OpenWRT and Supermicro

Wow! Here's one for the record books that took a bit of research for such a simple solution. Everything that's written here applies to an x86_64 install of OpenWRT on computer hardware with a video display as opposed to TTL / Serial output that some monitors have.

The Issue

Are you installing OpenWRT on a Supermicro, or posibly another x86_64 motherboard / platform with an ASPEED or equivalent IPMI (Intelligent Platform Management Interface) / BMC (Baseboard Management Controller that has multiple video outputs (VGA, HDMI, Display Port, etc.)?

Have you experienced an issue where the OpenWRT boot sequence starts, but then the screen goes black and nothing is displayed?

Well, read on for an explanation of the issue. If you want to take a shortcut and just solve the issue, then read the next section.

Shortcut, errr, The Solution

In the /boot/grub/grub.cfg file (remember, OpenWRT uses "Legacy" Grub, so all of the GRUB2 information out there has to be 'taken with a grain of salt' and translated into "Legacy" Grub);

• "Stock" / Original OpenWRT Boot / Kernel Commands / Parameters;
  • linux /boot/vmlinuz root=PARTUUID=d2e27698-02 rootfstype=squashfs rootwait console=ttyS0,115200n8 console=tty0 noinitrd
• Change it to this: (IE, just add nomodeset to the end of the line);
  • linux /boot/vmlinuz root=PARTUUID=d2e27698-02 rootfstype=squashfs rootwait console=ttyS0,115200n8 console=tty0 noinitrd nomodeset
• Hints Obtained from;
  • https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt )

...and that's it. The output will be displayed to the monitor hooked up to the VGA Port

One More Thing to Make Sure of

For the Supermicro Motherboards, make sure the below setting is configured in the BIOS Setup / Settings;

• For the X11SAE-F Motherboard (it may vary with different models);
  • Under Advanced, Chipset Configuration, North Bridge, Intel IGD Configuration, Primary Display (Auto, IGD, PCIE), set it to either: Auto or PCIE

In the System Block Diagram of the X11SAE-F Motherboard below, it shows the VGA port at the end of a PCIe lane.

System Block Diagram

Of course this begs the question about where the heck the connector for the IGD (Integrated Graphics Device) is. Apparently there isn't one, which may be involved with the implementation of the ASPEED chip. The block diagram doesn't explicitly show the HDMI or Display Port at the end of a PCIe lane, but they are.

So if you were thinking the VGA port (or the HDMI or Display Port Connectors) are the IGD, they're not. If one looks at the information on the Wikipedia Page (https://en.wikipedia.org/wiki/List_of_Intel_Pentium_processors) or the Intel Page (https://ark.intel.com/content/www/us/en/ark/products/87261/intel-pentium-processor-n3700-2m-cache-up-to-2-40-ghz.html), in indicates that N3700 CPU (really and SoC (System on a Chip)) has a CPU and GPU integrated into the same device with only HDMI and Display Port Outputs. That, combined with the fact (according the System Block Diagram) the VGA output seems to be coming from ASPEED chip, leads one to believe the Pentium based Braswell CPU in the N3700 SoC indeed has HD Graphics 400 or 405 capability that is simply not hooked up to anything.

So we're right back to the point of asking why the IGD choice is in the BIOS settings if there isn't a connector for the HD Graphics 400 / 405 built into the Pentium CPU? IE, behavior and documentation seems to indicate there are two GPUs in the N3700 SoC package. One GPU is built into the Pentium CPU and the other is a discrete GPU at the end of a couple of PCIe lanes in the SoC. This isn't stated explicitly, but all evidence seems to point to this conclusion. Except for what OpenWRT displays...

Using the LSPCI command in OpenWRT, the "VGA Port" is reported as "VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30)" Now that seems to indicate a separate Video Controller (separate from the GPU on the N3700, and the possible GPU built into the core of the Pentium CPU). Wow, that would be three different video controllers on this motherboard, all built into other chips. Still on the end of a PCIe lane. No wonder OpenWRT acts wonky with it's display.

Fallback Incase One Sets the Above Noted Primary Display to IGD

If the Primary Display is set to IGD in the BIOS settings (BIOS Setup / Settings: Advanced, Chipset Configuration, North Bridge, Intel IGD Configuration, Primary Display (Auto, IGD, PCIE)) it will result in a blank or black screen no matter which port the monitor is plugged into.

Solution? See below;

• Connect a monitor to the HDMI port.
• Set JPG1 Jumper to Disable VGA (Pins 1-2 are Enable, Pins 2-3 are Disable)

Why? Because disabling the VGA port has the effect of enabling the HDMI and Display Port connectors.

Remember, for the above mentioned Supermicro motherboard, do NOT conflate IGD and VGA as they are not equivalent. As previously noted, the hint was in the block diagram of the motherboard and involves the ASPEED IPMI / BMC AST2400 Chip. The diagram shows VGA connected to the ASPEED BMC AST 2400 chip, which in turn is shown connected to the CPU via a PCIe lane. Get it? VGA is at the end of a PCIe lane, so setting the video in the BIOS to PCIe outputs to the VGA and Display Port.

Why Not Just Disable the VGA Port with a Jumper (JPG1 on Supermicro Motherboards)

Well, bad news: If you do that, it'll work in that all the boot up messages will be displayed on a monitor hooked up to the HDMI or Display Port. But... And this is a big BUT: Nothing will be displayed on the iKVM/HTML5 (or the 'Remote Control' JAVA Console). That's kind of a nice thing to have working. See the above System Block Diagram and you'll see that the JPG1 jumper really seems to disable the VGA of the ASPEED chip.

Other Notes

Setting Video Settings with Kernel Command Line Parameters

In the /boot/grub/grub.cfg file, at the end of the "vmlinuz" line;

• vga=795 works (from https://en.wikipedia.org/wiki/VESA_BIOS_Extensions#Linux_video_mode_numbers)
• video=1280x1024 (doesn't work unless i915 kernel driver is enabled, which one doesn't want because the ASPEED iKVM capbability will break)

Note, when setting the vga=795 parameter, the first part of the boot sequence (first couple of seconds, including "Decompressing Linux") cannot be seen until it reaches the 'change video' part of the boot. IE, the below part;

[    1.486659] simple-framebuffer simple-framebuffer.0: framebuffer at 0x81000000, 0x500000 bytes, mapped to 0xffffc90001000000

[    1.497977] simple-framebuffer simple-framebuffer.0: format=a8r8g8b8, mode=1280x1024x32, linelength=5120

[    1.544395] Console: switching to colour frame buffer device 160x64

[    1.587467] simple-framebuffer simple-framebuffer.0: fb0: simplefb registered!

Changing Resolution After Booting

Good news, it doesn't seem to be possible to adjust resolution after booting using fbset or xrandr because they don't exist as packages in OpenWRT. Makes sense given that most routers don't freakin' have video cards! It looks like some people have compiled custom versions of busybox.

No real need to change the resolution, unless one wants to see the full lines of boot messages during boots. Keep in mind, it's available for review after booting using the DMESG Command. Of course if booting is a problem it might be helpful to see the entire line. An alternative would be to remove the nomodeset parameter from the grub.cfg file.

ASPEED's iKVM Capability when Rebooting

More good news, the ASPEED's iKVM capability doesn't work across reboots, so one has to close or refresh the IPMI / BMC web browser tab / window.

MODESETTING (AKA Kernel Modesetting (KMS))

nomodeset=0 and nomodeset seem to be equivalent.

Differences in Boot Log Items (DMESG);

With JPG1 Jumper Enabled;

[    1.830724] Non-volatile memory driver v1.3
[    1.835663] Linux agpgart interface v0.103
[    1.842052] [drm] Memory usable by graphics device = 2048M
[    1.847750] [drm] Replacing VGA console driver
[    1.862012] Console: switching to colour dummy device 80x25
[    1.868432] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    1.868438] [drm] Driver supports precise vblank timestamp query.
[    1.891463] [drm] Initialized i915 1.6.0 20170818 for 0000:00:02.0 on minor 0
[    1.899211] [drm] HDaudio controller not detected, using LPE audio instead
[    1.899211]
[    1.915025] loop: module loaded
[    1.918457] Guest personality initialized and is inactive
[    1.924095] VMCI host device registered (name=vmci, major=10, minor=63)
[    1.930796] Initialized host personality
[    1.935699] VMware PVSCSI driver - version 1.0.7.0-k

With JPG1 Jumper Disabled;

[    1.999013] Non-volatile memory driver v1.3
[    2.003980] Linux agpgart interface v0.103
[    2.012403] [drm] Memory usable by graphics device = 2048M
[    2.018099] [drm] Replacing VGA console driver
[    2.032397] Console: switching to colour dummy device 80x25
[    2.038920] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    2.038927] [drm] Driver supports precise vblank timestamp query.
[    2.060952] [drm] Initialized i915 1.6.0 20170818 for 0000:00:02.0 on minor 0
[    2.068660] [drm] HDaudio controller not detected, using LPE audio instead
[    2.068660]
[    2.084438] loop: module loaded
[    2.087882] Guest personality initialized and is inactive
[    2.093496] VMCI host device registered (name=vmci, major=10, minor=63)
[    2.100201] Initialized host personality
[    2.105116] VMware PVSCSI driver - version 1.0.7.0-k

With JPG1 Enabled and nomodeset paramter set in grub.cfg on the "vmlinuz line";

[    1.714978] Linux agpgart interface v0.103
[    1.726847] loop: module loaded
[    1.730427] Guest personality initialized and is inactive
[    1.736124] VMCI host device registered (name=vmci, major=10, minor=63)
[    1.742948] Initialized host personality
[    1.747960] VMware PVSCSI driver - version 1.0.7.0-k

NOMODESET, according to: https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt;

Disable kernel modesetting. Most systems' firmware sets up a display mode and provides framebuffer memory for output. With nomodeset, DRM and fbdev drivers will not load if they could possibly displace the pre-initialized output. Only the system framebuffer will be available for use. The respective drivers will not perform display-mode changes or accelerated rendering.

...and just FYI, because kernel.org is so good at explaining acronyms;

DRM = Direct Rendering Manager

FBDEV = Linux Frame Buffer ( https://en.wikipedia.org/wiki/Linux_framebuffer )

Last Tip(s)

It goes without saying: Upgrade to the latest BIOS, IPMI, etc. Firmware!

Links for More Information

• Kernel Command Line Paramters: https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt
• Video Settings: https://en.wikipedia.org/wiki/VESA_BIOS_Extensions#Linux_video_mode_numbers
• Example of GRUB 2 documentation that won't help (but demonstrates what GRUB 2 references look like, even if they don't specifically say it, so you know what to watch out for, and ignore, because OpenWRT uses "Legacy" GRUB): https://www.gnu.org/software/grub/manual/grub/html_node/Simple-configuration.html
• Credit to the hint that lead to the solution (even if it wasn't specifically addressing the issue, it was the impetus for the exploration on this subject): https://askubuntu.com/questions/716957/what-do-the-nomodeset-quiet-and-splash-kernel-parameters-mean