MV1000W along with Other OpenWRT Routers

...ever run into a situation where there's a desire to change how the "Disk Drive" of a device is organized?

Example

OK, take the GL.iNet brand GL-MV1000W (or just plain GL-MV1000). It has a onboard 8GB eMMC! That's like having an 8GB MicroSD Card welded to the PCB. Cool!

Except for one thing... It just isn't taken advantage of. The default configuration has the following;

• 16.3 MB allocated to the "/boot" Partition (IE, where the Image file containing the "InitRAMFS" stuff, and a .dtb file for other information)
• 512.3 MB allocated to the "/rom" Partition (in the 'shape' of a SquashFS Partition containing all the regular file system stuff, configuration files (/etc), .ko (kernel module files), etc.)
• 6.8 GB allocated to "/data" (an Overlay Partition)

lsblk

NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0          7:0    0   489M  0 loop 
mtdblock0     31:0    0   960K  0 disk 
mtdblock1     31:1    0    32K  0 disk 
mtdblock2     31:2    0    32K  0 disk 
mmcblk0      179:0    0   7.3G  0 disk 
├─mmcblk0p1  179:1    0  16.3M  0 part 
├─mmcblk0p2  179:2    0 512.3M  0 part 
└─mmcblk0p3  179:3    0   6.8G  0 part 
mmcblk0boot0 179:8    0     4M  1 disk 
mmcblk0boot1 179:16   0     4M  1 disk 

cfdisk /dev/mmcblk0

                                      Disk: /dev/mmcblk0
                      Size: 7.29 GiB, 7818182656 bytes, 15269888 sectors
                              Label: dos, identifier: 0x00000000

    Device             Boot          Start         End     Sectors      Size    Id Type
    /dev/mmcblk0p1     *              2048       35327       33280     16.3M    83 Linux
    /dev/mmcblk0p2                   36864     1085951     1049088    512.3M    83 Linux
    /dev/mmcblk0p3                 1087488    15274495    14187008      6.8G    83 Linux

All the "Linux stuff" is on mmcblk0 with the "stuff on the MTD device" being related to the Marvell U-Boot, boot loader.

It's all fine and cool, but... What if?

The Operating System

This idea came about for the simple reason that the rtl8192bu Wireless Radio does NOT work with the 23.x versions of OpenWRT. But it does work with the GL Provided 19.07.8 version of OpenWRT. There are BS instructions on the OpenWRT page for this router: https://openwrt.org/toh/gl.inet/gl-mv1000

Increasing the root filesystem
A simplified version of the extroot_configuration works. FIXME: contra main text, this works even though there's no overlay on `mtd` on the fresh install.

opkg update; opkg install block-mount cfdisk rsync
cfdisk /dev/mmcblk1, create a new primary partition on the large unused space. Take note of the device (e.g. /dev/mmcblk1p3).
mkfs.ext4 on your new partition (e.g. /dev/mmcblk1p3).
mkdir /mnt/bigroot; mount /dev/mmcblk1p3 /mnt/bigroot
block detect | uci import fstab. You now have eveything set on /etc/config/fstab.
copy the root filesystem to /mnt/bigroot (e.g. rsync -a -x / /mnt/bigroot).
edit /etc/config/fstab. change the old root / filesystem to mount on /rom, and the new one from /mnt/bigroot to /overlay.
reboot and check the mounts with the df command.
If the mounts are not available, please validate /etc/config/fstab and check if /rom and /overlay have option enabled set to 1 (https://forum.openwrt.org/t/guide-for-gl-inet-gl-mv1000-brume/89398/17).

Save, reboot and validate again with the df command.

The effort is appreciated, but the written implementation blows donkey snot. Plus the end result isn't nearly as good as what's here.

The Question

Can it be turned into a 'real computer'? IE, what about leaving the '/boot' Partition in place, but then having the rest of the eMMC card as a single large EXT4 partition?

Solution (Not)

Booting from an InitRAMFS.bin file (actually on the OpenWRT.org Download Website). Nope, doesn't seem to work. Several "bootm" and "booti" methods were attempted at the Marvell / U-Boot command prompt with the best outcome displaying "Starting kernel . . .", and then hanging there.

Solution (Yes, but the long way)

First, get a microSD card (doesn't have to be huge, and can be smaller than the 'built-in' eMMC device) and insert it into the card slot.

"Bulk Route"

If the microSD Card is 8GB or larger, just use the DD command: dd if=/dev/mmcblk0 of=/dev/mmcblk1 bs=16M conv=noerror,sync status=progress

Keep in mind, this produces an 'exact clone', same UUID numbers on Partitions, etc. So it can cause issues if the microSD Card is left in place.

Skip the rest of the items in the "Solutions (Yes...") Section

Prepare the microSD Card

wipefs -a /dev/mmcblk1 (should be /dev/mmcblk1, but double check using lsblk -f (and of course install lsblk if it isn't available, as well as for any other command noted in here that isn't available until being installed)

Then use CFDISK or FDISK to create the same Partitions from

Copy the "/boot" Partition

mkdir /mnt/extracted

unsquashfs -d /mnt/extracted /dev/mmcblk0p2

mksquashfs /mnt/extracted /dev/mmcblk1p2 -comp xz -noappend

It isn't covered here, but installing unsquashfs, mksquashfs, xz, etc. will all need to be installed using OPKG

And finally, from the Marvell U-Boot Command Line (press gl (that's g as in Girl and L as in little) in quick succession)

setenv bootargs console=ttyMV0,115200 earlycon=ar3700_uart,0xd0012000 root=/dev/mmcblk1p2 rw rootwait

load mmc 1:1 0x4ff0000 /armada-gl-mv1000-emmc.dtb

load mmc 1:1 0x5000000 /Image

booti 0x5000000 - 0x4ff0000

OK, done with the booting to the alternate boot device, now on to 'fixing up' the built in eMMC device

...running out of time today, so just putting the initial notes in place.

Assumes one is booted from the 'alternate boot device', IE mmcblk1, not mmcblk0.

• umount anything mounted on /dev/mmcblk0pX (happens if using DD and duplicate UUIDs are created)
• Get the contents of /dev/mmcblk0p2 (the original /rom (what OpenWRT refers to it as, since it is a Read Only SquashFS File system), AKA (in Rocky Linux parlance), the ROOT ( / ) Partition;
  • mkdir /mnt/extracted
  • unsquashfs -d /mnt/extracted /dev/mmcblk0p2
  • mksquashfs /mnt/extracted /dev/sda2 -comp xz -noappend
• Wipe out every Partition on mmcblk0, except the first Partition (/boot in Rocky Linux parlance, containing Image (the IntraRAMFS equivalent) and .dtb file)
  • use CFDISK
  • Resize the /boot partition if desired
    • e2fsck -f /dev/mmcblk0p1
    • resize2fs /dev/mmcblk0p1
  • ...then create a 6 GB (or whatever size desired) main partition
  • ...leaving a bit less than 1GB for a Swap Partition
• Copy everything from /mnt/extracted to /mnt/mmcblk0p2 (mount it first of course)
  • Optionally check the file system with e2fsck
• Copy all the custom settings from the Overlay "Partition"
  • rsync -av /rom/overlay/upper/ /mnt/mmcblk0p2/ (this will overwrite all the 'stock' files with duplicate or upgraded files)
• Reboot

Watch Out!

There is one insidious thing that will occur during boot and even after booting is complete (at around 5 minutes after booting)

During the boot process, when the Image file (which contains the kernel and mounts the root file system among other things) is loaded and run, it mounts /dev/mmcblk0p2 as /rom and /data. /data isn't a bit issue (but it is puzzling, will explain shortly). /rom is a concern, because it gets mounted as Read Only. Here's why it is a concern: /dev/mmcblk0p2 is the Root (IE / ) Partition and needs to be Read / Write. Now as originally configured it was a SquashFS File system, but now converted to EXT4.

By definition SquashFS is read only (unless one is building it, IE 'squashing' it), so why explicitly mount it as /rom Read Only? Why not just mount it, as it will be Read Only? Who knows, but that's how it is.

Solution?

in /etc/rc.local (or LuCI GUI);

sleep 11
umount /rom
umount /data
mount -o remount,rw /dev/mmcblk0p2

This solves the Read Only issue, but as mentioned, for some reason, /data remounts (but in Read / Write mode). Thankfully /rom does not. And since the /data remounting thing doesn't affect anything, it doesn't matter. It is frustrating, but what's even more frustrating is there is no logging evidence whatsoever that /data got remounted. Woof!

Check it with this command (before applying the fix): mount | grep /dev/mmcblk0p2

...and you'll get this;

/dev/mmcblk0p2 on /rom type ext4 (ro,noatime,data=ordered)
/dev/mmcblk0p2 on / type ext4 (ro,relatime,data=ordered)
/dev/mmcblk0p2 on /data type ext4 (ro,relatime,data=ordered)

Assuming the order displayed is also the order of how things were mounted, it appears that whoever mounts first trumps everything else in terms of Read / Only or Read / Write. Once /rom is unmounted, as noted above, / can be remounted as Read / Write with: mount -o remount,rw /dev/mmcblk0p2

Marvell U-Boot and PRINTENV

Below are the default values displayed by the printenv command from within the Marvell U-Boot Command Prompt;

args_mmc=setenv bootargs "${console} root=/dev/mmcblk0p2 rw rootwait"
args_sd=setenv bootargs "${console} root=/dev/mmcblk1p2 rw rootwait"
args_usb=setenv bootargs console=ttyMV0,115200 earlycon=ar3700_uart,0xd0012000 root=/dev/sda2 rw rootwait
baudrate=115200
boot_mmc=echo "   Booting from mmc ..."; run clear_args; run args_mmc; run load_mmc && booti ${kernel_addr} - ${fdt_addr}
boot_mtd=echo "   Booting from nor ..."; run clear_args; run load_mtd && bootm $kernel_addr - $fdt_addr
boot_sd=echo "   Booting from sd ..."; run clear_args; run args_sd; run load_sd && booti $kernel_addr - $fdt_addr
boot_tftp=echo "   Booting from tftp ..."; run clear_args; run load_tftp && booti $kernel_addr - $fdt_addr
boot_usb=echo "   Booting from usb ..."; run clear_args; run args_usb; run load_usb && booti $kernel_addr - $fdt_addr
bootargs=console=ttyMV0,115200 earlycon=ar3700_uart,0xd0012000
bootcmd=run bootscript; run boot_mmc; run boot_mtd; run boot_sd; run boot_usb; run boot_tftp
bootdelay=2
bootscript=load mmc ${mmc_dev}:${image_part} ${loadaddr} boot.scr && echo "   Running bootscript from mmc ..."; source ${loadaddr}
clear_args=setenv bootargs "${console}"
console=console=ttyMV0,115200 earlycon=ar3700_uart,0xd0012000
ethact=neta@30000
ethprime=eth0
fdt_addr=0x4ff0000
fdt_high=0xffffffffffffffff
fdt_name=armada-gl-mv1000-emmc.dtb
fdtcontroladdr=3f715098
image_dir=/
image_name=Image
image_part=1
image_ramfs=openwrt-mvebu-cortexa53-gl-mv1000-initramfs-kernel.bin
ipaddr=192.168.1.1
kernel_addr=0x5000000
lf=tftpboot openwrt-gl-mv1000.bin && mmc dev 0 && mmc erase 0x0 0x109800 && mmc write $kernel_addr 0x0 0x32000
load_mmc=load mmc ${mmc_dev}:${image_part} ${fdt_addr} ${image_dir}${fdt_name} && load mmc ${mmc_dev}:${image_part} ${kernel_addr} ${image_dir}${image_name}
load_mtd=sf probe; sf read 0x4ff0000 0x100000 0x400000
load_sd=load mmc ${sd_dev}:${image_part} ${fdt_addr} ${image_dir}${fdt_name} && load mmc ${sd_dev}:${image_part} ${kernel_addr} ${image_dir}${image_name}
load_tftp=tftpboot $fdt_addr ${image_dir}${fdt_name} && tftpboot $kernel_addr ${image_dir}${image_ramfs}
load_usb=usb start; load usb 0:${image_part} ${fdt_addr} ${image_dir}${fdt_name} && load usb 0:${image_part} ${kernel_addr} ${image_dir}${image_name}
loadaddr=0x5000000
lu=bubt flash-image.bin spi tftp
mmc_dev=0
netdev=eth0
netmask=255.255.255.0
sd_dev=1
serverip=192.168.1.2
stderr=serial@12000
stdin=serial@12000
stdout=serial@12000

Environment size: 2519/32764 bytes

Some Bonus MMC Commands for U-Boot

mmc list

mmc dev WhatEverDeviceNumber

mmc part

mmc info