0

We have an installation with Mailcow on AWS EC2. The volume earlier had 40GB which was modified to 60GB when we saw df giving 100% usage.

After the volume size was increased, the changes did not reflect so we restarted the server (we generally do that in GCP and it orks fine).

After rebooting the server, the server did not boot at all.

The Instance log in AWS said - No space left on device.

So we detached and connected this Volume to another server as secondary storage and copied some files to another "Backup" volume so that we restore the files once the main server starts.

Now, we did umount and attached the old volume to the main server and we are getting the following Instance log.

SSH can still not connect.

[ 0.000000] Linux version 5.4.0-1038-aws (buildd@lgw01-amd64-037) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #40~18.04.1-Ubuntu SMP Sat Feb 6 01:56:56 UTC 2021 (Ubuntu 5.4.0-1038.40~18.04.1-aws 5.4.86) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-1038-aws root=UUID=e93344bc-6cac-47fe-b487-5747fd2f1c39 ro console=tty1 console=ttyS0 nvme_core.io_timeout=4294967295 [ 0.000000] KERNEL supported cpus: [ 0.000000]   Intel GenuineIntel [ 0.000000]   AMD AuthenticAMD [ 0.000000]   Hygon HygonGenuine [ 0.000000]   Centaur CentaurHauls [ 0.000000]   zhaoxin   Shanghai   [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000efffffff] usable [ 0.000000] BIOS-e820: [mem 0x00000000fc000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000010fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.7 present. [ 0.000000] DMI: Xen HVM domU, BIOS 4.2.amazon 08/24/2006 [ 0.000000] Hypervisor detected: Xen HVM [ 0.000000] Xen version 4.2. [ 0.000000] Netfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated NICs. [ 0.000000] Blkfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated disks. [ 0.000000] You might have to change the root device
 [ 0.000000] from /dev/hd[a-d] to /dev/xvd[a-d] [ 0.000000] in your root= kernel command line option [ 0.015246] tsc: Fast TSC calibration using PIT [ 0.015250] tsc: Detected 2299.922 MHz processor [ 0.015251] tsc: Detected 2300.042 MHz TSC [ 0.016281] last_pfn = 0x110000 max_arch_pfn = 0x400000000 [ 0.016329] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT   [ 0.016337] last_pfn = 0xf0000 max_arch_pfn = 0x400000000 [ 0.023423] found SMP MP-table at [mem 0x000fbc50-0x000fbc5f] [ 0.023502] check: Scanning 1 areas for low memory corruption [ 0.023781] RAMDISK: [mem 0x3580f000-0x36bfefff] [ 0.023875] ACPI: Early table checksum verification disabled [ 0.023880] ACPI: RSDP 0x00000000000EA020 000024 (v02 Xen   ) [ 0.023883] ACPI: XSDT 0x00000000FC00E2A0 000054 (v01 Xen HVM   00000000 HVML 00000000) [ 0.023888] ACPI: FACP 0x00000000FC00DF60 0000F4 (v04 Xen HVM   00000000 HVML 00000000) [ 0.023894] ACPI: DSDT 0x00000000FC0021C0 00BD19 (v02 Xen HVM   00000000 INTL 20090123) [ 0.023897] ACPI: FACS 0x00000000FC002180 000040 [ 0.023899] ACPI: FACS 0x00000000FC002180 000040 [ 0.023901] ACPI: APIC 0x00000000FC00E060 0000D8 (v02 Xen HVM   00000000 HVML 00000000) [ 0.023904] ACPI: HPET 0x00000000FC00E1B0 000038 (v01 Xen HVM   00000000 HVML 00000000) [ 0.023906] ACPI: WAET 0x00000000FC00E1F0 000028 (v01 Xen HVM   00000000 HVML 00000000) [ 0.023908] ACPI: SSDT 0x00000000FC00E220 000031 (v02 Xen HVM   00000000 INTL 20090123) [ 0.023911] ACPI: SSDT 0x00000000FC00E260 000033 (v02 Xen HVM   00000000 INTL 20090123) [ 0.026956] No NUMA configuration found [ 0.026958] Faking a node at [mem 0x0000000000000000-0x000000010fffffff] [ 0.026968] NODE_DATA(0) allocated [mem 0x10ffd3000-0x10fffdfff] [ 0.027191] Zone ranges: [ 0.027192]   DMA   [mem 0x0000000000001000-0x0000000000ffffff] [ 0.027193]   DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.027194]   Normal   [mem 0x0000000100000000-0x000000010fffffff] [ 0.027195]   Device   empty [ 0.027196] Movable zone start for each node [ 0.027199] Early memory node ranges [ 0.027200]   node   0: [mem 0x0000000000001000-0x000000000009dfff] [ 0.027201]   node   0: [mem 0x0000000000100000-0x00000000efffffff] [ 0.027203]   node   0: [mem 0x0000000100000000-0x000000010fffffff] [ 0.027206] Zeroed struct page in unavailable ranges: 99 pages [ 0.027207] Initmem setup node 0 [mem 0x0000000000001000-0x000000010fffffff] [ 0.047301] ACPI: PM-Timer IO Port: 0xb008 [ 0.047363] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-47 [ 0.047366] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.047367] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 low level) [ 0.047368] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 low level) [ 0.047369] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 low level) [ 0.047375] Using ACPI (MADT) for SMP configuration information [ 0.047377] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.047382] TSC deadline timer available [ 0.047388] smpboot: Allowing 15 CPUs, 13 hotplug CPUs [ 0.047399] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
 [ 0.047400] PM: Registered nosave memory: [mem 0x0009e000-0x0009ffff] [ 0.047401] PM: Registered nosave memory: [mem 0x000a0000-0x000dffff] [ 0.047402] PM: Registered nosave memory: [mem 0x000e0000-0x000fffff] [ 0.047403] PM: Registered nosave memory: [mem 0xf0000000-0xfbffffff] [ 0.047404] PM: Registered nosave memory: [mem 0xfc000000-0xffffffff] [ 0.047406] [mem 0xf0000000-0xfbffffff] available for PCI devices [ 0.047407] Booting paravirtualized kernel on Xen HVM [ 0.047410] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns [ 0.047416] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:15 nr_cpu_ids:15 nr_node_ids:1 [ 0.048425] percpu: Embedded 54 pages/cpu s184320 r8192 d28672 u262144 [ 0.048460] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.048466] Built 1 zonelists, mobility grouping on.  Total pages: 1032072 [ 0.048467] Policy zone: Normal [ 0.048469] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-1038-aws root=UUID=e93344bc-6cac-47fe-b487-5747fd2f1c39 ro console=tty1 console=ttyS0 nvme_core.io_timeout=4294967295 [ 0.049333] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) [ 0.049729] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.049862] mem auto-init: stack:off, heap alloc:on, heap free:off [ 0.075032] Memory: 3999508K/4193908K available (14339K kernel code, 2313K rwdata, 4608K rodata, 2652K init, 5140K bss, 194400K reserved, 0K cma-reserved) [ 0.075042] random: get_random_u64 called from __kmem_cache_create+0x41/0x550 with crng_init=0 [ 0.075233] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=15, Nodes=1 [ 0.075249] Kernel/User page tables isolation: enabled [ 0.075267] ftrace: allocating 42773 entries in 168 pages [ 0.091878] rcu: Hierarchical RCU implementation. [ 0.091880] rcu:   RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=15. [ 0.091882]    Tasks RCU enabled. [ 0.091883] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.091884] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=15 [ 0.094396] NR_IRQS: 524544, nr_irqs: 952, preallocated irqs: 16 [ 0.094438] xen:events: Using 2-level ABI [ 0.094485] xen:events: Xen HVM callback vector for event delivery is enabled [ 0.094650] random: crng done (trusting CPU's manufacturer) [ 0.109716] Console: colour VGA+ 80x25 [ 0.289009] printk: console [tty1] enabled [ 0.290231] Cannot get hvm parameter CONSOLE_EVTCHN (18): -22! [ 1.260724] printk: console [ttyS0] enabled [ 1.273680] ACPI: Core revision 20190816 [ 1.284938] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 30580167144 ns [ 1.341860] APIC: Switch to symmetric I/O mode setup [ 1.349513] x2apic: IRQ remapping doesn't support X2APIC mode
 [ 1.358640] Switched APIC routing to physical flat.
 [ 1.367001] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0
 [ 1.380660] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x21275cd9627, max_idle_ns: 440795323243 ns
 [ 1.398650] Calibrating delay loop (skipped), value calculated using timer frequency.. 4600.08 BogoMIPS (lpj=9200168)
 [ 1.426652] pid_max: default: 32768 minimum: 301
 [ 1.438711] LSM: Security Framework initializing
 [ 1.454675] Yama: becoming mindful.
 [ 1.474752] AppArmor: AppArmor initialized
 [ 1.482716] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
 [ 1.514665] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
 [ 1.530683] *** VALIDATE tmpfs ***
 [ 1.538836] *** VALIDATE proc ***
 [ 1.554745] *** VALIDATE cgroup1 ***
 [ 1.558652] *** VALIDATE cgroup2 ***
 [ 1.574789] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
 [ 1.582649] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
 [ 1.602656] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
 [ 1.614651] Spectre V2 : Mitigation: Full generic retpoline
 [ 1.618649] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
 [ 1.626649] Speculative Store Bypass: Vulnerable
 [ 1.634653] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
 [ 1.638850] Freeing SMP alternatives memory: 40K
 [ 1.648636] clocksource: xen: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
 [ 1.658674] installing Xen timer for CPU 0
 [ 1.662734] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz (family: 0x6, model: 0x4f, stepping: 0x1)
 [ 1.666693] cpu 0 spinlock event irq 53
 [ 1.670860] Performance Events: unsupported p6 CPU model 79 no PMU driver, software events only.
 [ 1.674741] rcu: Hierarchical SRCU implementation.
 [ 1.679374] NMI watchdog: Perf NMI watchdog permanently disabled
 [ 1.682920] smp: Bringing up secondary CPUs ...
 [ 1.686795] installing Xen timer for CPU 1
 [ 1.690742] x86: Booting SMP configuration:
 [ 1.694662] .... node  #0, CPUs:  #1
 [ 1.695330] cpu 1 spinlock event irq 59
 [ 1.710696] smp: Brought up 1 node, 2 CPUs
 [ 1.714657] smpboot: Max logical packages: 8
 [ 1.718654] smpboot: Total of 2 processors activated (9200.16 BogoMIPS)
 [ 1.723021] devtmpfs: initialized
 [ 1.726725] x86/mm: Memory block size: 128MB
 [ 1.735136] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
 [ 1.738727] futex hash table entries: 4096 (order: 6, 262144 bytes, linear)
 [ 1.742799] pinctrl core: initialized pinctrl subsystem
 [ 1.746821] PM: RTC time: 02:39:14, date: 2021-03-13
 [ 1.750878] NET: Registered protocol family 16
 [ 1.754796] audit: initializing netlink subsys (disabled)
 [ 1.758699] audit: type=2000 audit(1615603154.643:1): state=initialized audit_enabled=0 res=1
 [ 1.782752] EISA bus registered
 [ 1.786665] cpuidle: using governor ladder
 [ 1.790670] cpuidle: using governor menu
 [ 1.794725] ACPI: bus type PCI registered
 [ 1.798654] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
 [ 1.803246] PCI: Using configuration type 1 for base access
 [ 1.808151] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
 [ 1.810748] ACPI: Added _OSI(Module Device)
 [ 1.814656] ACPI: Added _OSI(Processor Device)
 [ 1.818656] ACPI: Added _OSI(3.0 _SCP Extensions)
 [ 1.822657] ACPI: Added _OSI(Processor Aggregator Device)
 [ 1.826658] ACPI: Added _OSI(Linux-Dell-Video)
 [ 1.830654] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
 [ 1.834654] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
 [ 1.842858] ACPI: 3 ACPI AML tables successfully acquired and loaded
 [ 1.848419] ACPI: Interpreter enabled
 [ 1.850667] ACPI: (supports S0 S4 S5)
 [ 1.854654] ACPI: Using IOAPIC for interrupt routing
 [ 1.858707] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
 [ 1.863123] ACPI: Enabled 2 GPEs in block 00 to 0F
 [ 1.915723] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
 [ 1.918665] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3]
 [ 1.922685] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
 [ 1.927928] acpiphp: Slot [0] registered
 [ 1.932061] acpiphp: Slot [3] registered
 [ 1.935202] acpiphp: Slot [4] registered
 [ 1.939340] acpiphp: Slot [5] registered
 [ 1.943198] acpiphp: Slot [6] registered
 [ 1.947217] acpiphp: Slot [7] registered
 [ 1.951136] acpiphp: Slot [8] registered
 [ 1.955171] acpiphp: Slot [9] registered
 [ 1.959175] acpiphp: Slot [10] registered
 [ 1.963465] acpiphp: Slot [11] registered
 [ 1.967252] acpiphp: Slot [12] registered
 [ 1.971150] acpiphp: Slot [13] registered
 [ 1.975206] acpiphp: Slot [14] registered
 [ 1.979179] acpiphp: Slot [15] registered
 [ 1.983346] acpiphp: Slot [16] registered
 [ 1.987323] acpiphp: Slot [17] registered
 [ 1.991373] acpiphp: Slot [18] registered
 [ 1.995389] acpiphp: Slot [19] registered
 [ 1.999361] acpiphp: Slot [20] registered
 [ 2.003943] acpiphp: Slot [21] registered
 [ 2.007492] acpiphp: Slot [22] registered
 [ 2.011630] acpiphp: Slot [23] registered
 [ 2.016021] acpiphp: Slot [24] registered
 [ 2.019790] acpiphp: Slot [25] registered
 [ 2.023278] acpiphp: Slot [26] registered
 [ 2.027243] acpiphp: Slot [27] registered
 [ 2.031232] acpiphp: Slot [28] registered
 [ 2.037433] acpiphp: Slot [29] registered
 [ 2.039286] acpiphp: Slot [30] registered
 [ 2.043157] acpiphp: Slot [31] registered
 [ 2.047155] PCI host bridge to bus 0000:00
 [ 2.050656] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
 [ 2.054659] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
 [ 2.058656] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
 [ 2.062661] pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfbffffff window]
 [ 2.066661] pci_bus 0000:00: root bus resource [bus 00-ff]
 [ 2.070984] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
 [ 2.082932] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
 [ 2.092931] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
 [ 2.097287] pci 0000:00:01.1: reg 0x20: [io  0xc100-0xc10f]
 [ 2.099536] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
 [ 2.102657] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
 [ 2.106659] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
 [ 2.110659] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
 [ 2.115923] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
 [ 2.118704] * Found PM-Timer Bug on the chipset. Due to workarounds for a bug,
 [ 2.118704] * this clock source is slow. Consider trying other clock sources
 [ 2.125286] pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
 [ 2.128295] pci 0000:00:02.0: [1013:00b8] type 00 class 0x030000
 [ 2.131563] pci 0000:00:02.0: reg 0x10: [mem 0xf0000000-0xf1ffffff pref]
 [ 2.135260] pci 0000:00:02.0: reg 0x14: [mem 0xf3000000-0xf3000fff]
 [ 2.152314] pci 0000:00:03.0: [5853:0001] type 00 class 0xff8000
 [ 2.155614] pci 0000:00:03.0: reg 0x10: [io  0xc000-0xc0ff]
 [ 2.159102] pci 0000:00:03.0: reg 0x14: [mem 0xf2000000-0xf2ffffff pref]
 [ 2.182803] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11)
 [ 2.187021] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
 [ 2.190977] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
 [ 2.195012] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11)
 [ 2.229898] xen:balloon: Initialising balloon driver
 [ 2.258729] iommu: Default domain type: Translated 
 [ 2.262839] SCSI subsystem initialized
 [ 2.266748] pci 0000:00:02.0: vgaarb: setting as boot VGA device
 [ 2.270645] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
 [ 2.270660] pci 0000:00:02.0: vgaarb: bridge control possible
 [ 2.274653] vgaarb: loaded
 [ 2.278693] ACPI: bus type USB registered
 [ 2.282686] usbcore: registered new interface driver usbfs
 [ 2.286669] usbcore: registered new interface driver hub
 [ 2.290705] usbcore: registered new device driver usb
 [ 2.294708] pps_core: LinuxPPS API ver. 1 registered
 [ 2.298653] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
 [ 2.302669] PTP clock support registered
 [ 2.306748] EDAC MC: Ver: 3.0.0
 [ 2.311619] PCI: Using ACPI for IRQ routing
 [ 2.315700] NetLabel: Initializing
 [ 2.318652] NetLabel:  domain hash size = 128
 [ 2.322652] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
 [ 2.326679] NetLabel:  unlabeled traffic allowed by default
 [ 2.330721] hpet: 3 channels of 0 reserved for per-cpu timers
 [ 2.334665] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
 [ 2.338645] hpet0: 3 comparators, 64-bit 62.500000 MHz counter
 [ 2.360745] clocksource: Switched to clocksource xen
 [ 2.386057] *** VALIDATE bpf ***
 [ 2.390920] VFS: Disk quotas dquot_6.6.0
 [ 2.409680] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
 [ 2.420986] *** VALIDATE ramfs ***
 [ 2.426045] *** VALIDATE hugetlbfs ***
 [ 2.438521] AppArmor: AppArmor Filesystem Enabled
 [ 2.447548] pnp: PnP ACPI init
 [ 2.459258] system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved
 [ 2.479277] system 00:01: [io  0x08a0-0x08a3] has been reserved
 [ 2.492226] system 00:01: [io  0x0cc0-0x0ccf] has been reserved
 [ 2.501971] system 00:01: [io  0x04d0-0x04d1] has been reserved
 [ 2.510314] system 00:07: [io  0x10c0-0x1141] has been reserved
 [ 2.523370] system 00:07: [io  0xb044-0xb047] has been reserved
 [ 2.602841] pnp: PnP ACPI: found 8 devices
 [ 2.613633] thermal_sys: Registered thermal governor 'fair_share'
 [ 2.613634] thermal_sys: Registered thermal governor 'bang_bang'
 [ 2.623362] thermal_sys: Registered thermal governor 'step_wise'
 [ 2.630499] thermal_sys: Registered thermal governor 'user_space'
 [ 2.638305] thermal_sys: Registered thermal governor 'power_allocator'
 [ 2.653849] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
 [ 2.680719] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
 [ 2.688663] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
 [ 2.696506] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
 [ 2.704746] pci_bus 0000:00: resource 7 [mem 0xf0000000-0xfbffffff window]
 [ 2.718225] NET: Registered protocol family 2
 [ 2.724577] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
 [ 2.743427] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
 [ 2.758694] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
 [ 2.768790] TCP: Hash tables configured (established 32768 bind 32768)
 [ 2.776438] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
 [ 2.791273] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
 [ 2.800473] NET: Registered protocol family 1
 [ 2.815972] NET: Registered protocol family 44
 [ 2.825831] pci 0000:00:01.0: PIIX3: Enabling Passive Release
 [ 2.835268] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
 [ 2.845580] pci 0000:00:00.0: quirk_natoma+0x0/0x30 took 10063 usecs
 [ 2.856859] pci 0000:00:01.0: Activating ISA DMA hang workarounds
 [ 2.866589] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
 [ 2.877250] pci 0000:00:02.0: pci_fixup_video+0x0/0x110 took 10645 usecs
 [ 2.884906] PCI: CLS 0 bytes, default 64
 [ 2.890526] Trying to unpack rootfs image as initramfs...
 [ 3.187819] Freeing initrd memory: 20416K
 [ 3.193876] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
 [ 3.202458] software IO TLB: mapped [mem 0xec000000-0xf0000000] (64MB)
 [ 3.211301] check: Scanning for low memory corruption every 60 seconds
 [ 3.220797] Initialise system trusted keyrings
 [ 3.226394] Key type blacklist registered
 [ 3.231350] workingset: timestamp_bits=36 max_order=20 bucket_order=0
 [ 3.239367] zbud: loaded
 [ 3.243266] squashfs: version 4.0 (2009/01/31) Phillip Lougher
 [ 3.250171] fuse: init (API version 7.31)
 [ 3.255590] *** VALIDATE fuse ***
 [ 3.260112] *** VALIDATE fuse ***
 [ 3.264656] Platform Keyring initialized
 [ 3.273489] Key type asymmetric registered
 [ 3.278857] Asymmetric key parser 'x509' registered
 [ 3.284887] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 244)
 [ 3.293915] io scheduler mq-deadline registered
 [ 3.305803] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
 [ 3.318256] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
 [ 3.326673] ACPI: Power Button [PWRF]
 [ 3.331147] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
 [ 3.341499] ACPI: Sleep Button [SLPF]
 [ 3.347022] xen:grant_table: Grant tables using version 1 layout
 [ 3.359393] Grant table initialized
 [ 3.379007] Cannot get hvm parameter CONSOLE_EVTCHN (18): -22!
 [ 3.385918] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
 [ 3.440492] 00:06: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
 [ 3.451361] Linux agpgart interface v0.103
 [ 3.460380] loop: module loaded
 [ 3.464377] Invalid max_queues (4), will use default max: 2.
 [ 3.482134] scsi host0: ata_piix
 [ 3.486753] scsi host1: ata_piix
 [ 3.493135] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc100 irq 14
 [ 3.507306] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc108 irq 15
 [ 3.515713] libphy: Fixed MDIO Bus: probed
 [ 3.521304] tun: Universal TUN/TAP device driver, 1.6
 [ 3.527239] PPP generic driver version 2.4.2
 [ 3.532766] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
 [ 3.541242] ehci-pci: EHCI PCI platform driver
 [ 3.547891] ehci-platform: EHCI generic platform driver
 [ 3.557494] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
 [ 3.581987] ohci-pci: OHCI PCI platform driver
 [ 3.591174] ohci-platform: OHCI generic platform driver
 [ 3.617507] uhci_hcd: USB Universal Host Controller Interface driver
 [ 3.640319] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
 [ 3.680590] serio: i8042 KBD port at 0x60,0x64 irq 1
 [ 3.697291] serio: i8042 AUX port at 0x60,0x64 irq 12
 [ 3.710092] mousedev: PS/2 mouse device common for all mice
 [ 3.733957] rtc_cmos 00:02: registered as rtc0
 [ 3.734507] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
 [ 3.743892] rtc_cmos 00:02: alarms up to one day, 114 bytes nvram, hpet irqs
 [ 3.760420] blkfront: xvda: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled;
 [ 3.771842] device-mapper: uevent: version 1.0.3
 [ 3.801724] device-mapper: ioctl: 4.41.0-ioctl (2019-09-16) initialised: [email protected]
 [ 3.816373] platform eisa.0: Probing EISA bus 0
 [ 3.817030]  xvda: xvda1
 [ 3.825234] platform eisa.0: EISA: Cannot allocate resource for mainboard
 [ 3.825237] platform eisa.0: Cannot allocate resource for EISA slot 1
 [ 3.872539] platform eisa.0: Cannot allocate resource for EISA slot 2
 [ 3.881980] platform eisa.0: Cannot allocate resource for EISA slot 3
 [ 3.904135] platform eisa.0: Cannot allocate resource for EISA slot 4
 [ 3.921636] platform eisa.0: Cannot allocate resource for EISA slot 5
 [ 3.939743] platform eisa.0: Cannot allocate resource for EISA slot 6
 [ 3.948966] platform eisa.0: Cannot allocate resource for EISA slot 7
 [ 3.965999] platform eisa.0: Cannot allocate resource for EISA slot 8
 [ 3.975302] platform eisa.0: EISA: Detected 0 cards
 [ 3.984697] intel_pstate: CPU model not supported
 [ 3.991857] drop_monitor: Initializing network drop monitor service
 [ 4.023212] NET: Registered protocol family 10
 [ 4.034824] Segment Routing with IPv6
 [ 4.039896] NET: Registered protocol family 17
 [ 4.045629] Key type dns_resolver registered
 [ 4.053041] RAS: Correctable Errors collector initialized.
 [ 4.059633] IPI shorthand broadcast: enabled
 [ 4.064990] sched_clock: Marking stable (2816937817, 1248034260)->(4830929890, -765957813)
 [ 4.079652] registered taskstats version 1
 [ 4.085196] Loading compiled-in X.509 certificates
 [ 4.091990] Loaded X.509 cert 'Build time autogenerated kernel key: db012c1f1db57bdbe2ddf71b095981660ff04991'
 [ 4.112608] zswap: loaded using pool lzo/zbud
 [ 4.118301] Key type ._fscrypt registered
 [ 4.127994] Key type .fscrypt registered
 [ 4.136827] Key type big_key registered
 [ 4.143260] Key type encrypted registered
 [ 4.148597] AppArmor: AppArmor sha1 policy hashing enabled
 [ 4.159777] ima: No TPM chip found, activating TPM-bypass!
 [ 4.168314] ima: Allocated hash algorithm: sha1
 [ 4.177695] ima: No architecture policies found
 [ 4.183473] evm: Initialising EVM extended attributes:
 [ 4.194840] evm: security.selinux
 [ 4.199890] evm: security.SMACK64
 [ 4.209437] evm: security.SMACK64EXEC
 [ 4.229187] evm: security.SMACK64TRANSMUTE
 [ 4.238976] evm: security.SMACK64MMAP
 [ 4.248232] evm: security.apparmor
 [ 4.271341] evm: security.ima
 [ 4.275640] evm: security.capability
 [ 4.280316] evm: HMAC attrs: 0x1
 [ 4.285305] tsc: Refined TSC clocksource calibration: 2300.000 MHz
 [ 4.285591] xenbus_probe_frontend: Device with no driver: device/vif/0
 [ 4.292755] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x212735223b2, max_idle_ns: 440795277976 ns
 [ 4.329312] PM:   Magic number: 13:299:662
 [ 4.334180] rtc_cmos 00:02: setting system clock to 2021-03-13T02:39:17 UTC (1615603157)
 [ 4.355117] Freeing unused decrypted memory: 2040K
 [ 4.361238] Freeing unused kernel image memory: 2652K
 [ 4.382724] Write protecting the kernel read-only data: 22528k
 [ 4.390685] Freeing unused kernel image memory: 2008K
 [ 4.398511] Freeing unused kernel image memory: 1536K
 [ 4.415908] x86/mm: Checked W+X mappings: passed, no W+X pages found.
 [ 4.435493] x86/mm: Checking user space page tables
 [ 4.462292] x86/mm: Checked W+X mappings: passed, no W+X pages found.
 [ 4.474459] Run /init as init process

Loading, please wait...

starting version 237
 [ 4.621899] cryptd: max_cpu_qlen set to 1000
 [ 4.631628] xen_netfront: Initialising Xen virtual ethernet driver
 [ 4.654748] AVX2 version of gcm_enc/dec engaged.
 [ 4.665843] AES CTR mode by8 optimization enabled

Begin: Loading essential drivers ... [ 6.550708] raid6: avx2x4   gen() 23897 MB/s
 [ 6.602709] raid6: avx2x4   xor() 14499 MB/s
 [ 6.654709] raid6: avx2x2   gen() 20293 MB/s
 [ 6.710708] raid6: avx2x2   xor() 12233 MB/s
 [ 6.766710] raid6: avx2x1   gen() 17738 MB/s
 [ 6.822700] raid6: avx2x1   xor() 11671 MB/s
 [ 6.878710] raid6: sse2x4   gen() 13006 MB/s
 [ 6.930708] raid6: sse2x4   xor()  8263 MB/s
 [ 6.986711] raid6: sse2x2   gen() 10981 MB/s
 [ 7.038707] raid6: sse2x2   xor()  6972 MB/s
 [ 7.090713] raid6: sse2x1   gen()  9245 MB/s
 [ 7.142708] raid6: sse2x1   xor()  6370 MB/s
 [ 7.148352] raid6: using algorithm avx2x4 gen() 23897 MB/s
 [ 7.158946] raid6: .... xor() 14499 MB/s, rmw enabled
 [ 7.168480] raid6: using avx2x2 recovery algorithm
 [ 7.181754] xor: automatically using best checksumming function   avx    
 [ 7.207927] async_tx: api initialized (async)

done.

Begin: Running /scripts/init-premount ... done.

Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.

Begin: Running /scripts/local-premount ... [ 7.280495] Btrfs loaded, crc32c=crc32c-intel

Scanning for Btrfs filesystems

done.

Warning: fsck not present, so skipping root file system
 [ 7.320048] EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)

done.


Full log file at >> https://cdn.kkewash.com/i-00c8733e8541a29e6.log

As we can see that the Storage volume has mounted, so that's not the issue.

Is there anybody who has encountered the same issue? Any help would be much appreciated.

0

1 Answer 1

0

If it helps someone. The problem with MailCow started when the disk space got filled up due to Incoming emails. It might be due to a Spam blast that took the server space. Due to this, the OS which was Linux was not able to Boot. I could not connect even with SSH. Turns out, that, even to resize the disk, you need some free space on the disk which was not present.

The following steps were followed to rectify the issue :

  1. The Disk was detached from Old Server. I also created a snapshot of this disk before proceeding further.

  2. A new instance on AWS was created. Ubuntu was installed and a fresh installation of Mailcow was done. The main disk this time was much bigger than the old server.

  3. Now the Old Server Disk was attached to the New Server and mounted so that the old files could be copied to the new installation.

  4. Further, the steps mentioned in Mailcow Documentation were followed. https://mailcow.github.io/mailcow-dockerized-docs/i_u_m_migration/

Feel free to contact me if someone needs any help in this matter. I scratched my head for 3 days and nights before getting it fixed. I practically tried everything :).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.