OpenWRT Build System for CentOS 7

How does one create a "build system" (AKA Cross Compile / Cross-Compile / CrossCompile Build System) for OpenWRT?

Other Tutorials

• General Guide for building firmware: https://openwrt.org/docs/guide-user/additional-software/beginners-build-guide
  • Start of detailed Information: https://openwrt.org/docs/guide-developer/build-system/start
  • Pre-requisites for a : https://openwrt.org/docs/guide-developer/build-system/install-buildsystem
  • Some additional granular detail for working with specific versions: https://openwrt.org/docs/guide-developer/build-system/use-buildsystem
• Build environment tips: https://openwrt.org/docs/guide-developer/env
• SDK (for compiling specific packages): https://openwrt.org/docs/guide-developer/using_the_sdk
  • URL for SDK (look towards the bottom of the page for SDK): https://downloads.openwrt.org/releases/19.07.3/targets/mvebu/cortexa9/
• Third Party Site with nice information (Old, but this article itself notes even older OpenWRT documentation): http://www.acme-dot.com/building-openwrt-14-07-barrier-breaker-on-ubuntu-and-os-x/ (Don't follow the instructions as it GIT won't work anymore with non-SSL URLs)
• Entware Tips (The don't apply directly, but have some good tips if read from the context of OpenWRT)
  • Native Compilation: https://github.com/Entware/Entware-ng/wiki/Using-gcc-%28native-compilation%29
  • Compiling Packages from Source: https://github.com/Entware/Entware-ng/wiki/Compile-packages-from-sources
  • Creating Perl modules: https://github.com/Entware/Entware-ng/wiki/Self-installation-of-perl-modules

Environment

• Build System for Cross-Compiling: CentOS 7.8 (There seem to be some missing packages in CentOS 8, so CentOS 7.8 was chosen)
• OpenWRT 19.07.3 (Linux Kernel 4.14.180)
• Linksys WRT3200ACM Platform Information (see here for additional information)
  • Marvell Armada 385 88F6820 SoS (System on a Chip)
  • Manufacturer: Marvell, MVEBU (Marvell Embedded Business Unit)
  • Math Co-Processor: VFPv3-d16
  • CPU Information
    • ARM Family: Cortex-A (32-bit)
    • ARM Architecture: ARMv7-A
    • ARM Core: Cortex-A9
    • OpenWRT Target and Package Details
      • MVEBU (Marvell Embedded Business Unit)
        • Targets Available: cortexa53, cortexa72, cortexa9
        • Marvell Armada 37x/38x/XP (Armada 3700LP and Armada 7k/8k are 64 bit versions)
    • Equivalent DD-WRT / Entware Target and Package reference
      • ARMv7sf-k3.2: ARM version 7 with linux kernel 3.2
  • Sources:
    • https://en.wikipedia.org/wiki/List_of_ARM_microarchitectures
    • https://en.wikipedia.org/wiki/ARM_Cortex-A9
    • http://bin.entware.net/mipselsf-k3.4/

Steps (as of 9.2020 for the above Environment)

All of the below assumes one is storing files in the user's home directory.

OpenWRT recommends NOT compiling as a root users. To bypass this restriction;

• export FORCE_UNSAFE_CONFIGURE=1 (the export command adds the variable to the current user session, it will not persist if logging off, confirm with printenv or export -p)

Obtain Source Files from OpenWRT

• To clone into the current directory: git clone -b 19.07.3 https://github.com/openwrt/openwrt.git (downloads the 19.07 branch of OpenWRT, which also includes the latest version, 19.07.3)

• To clone into a specific directory: git clone -b 19.07.3 https://github.com/openwrt/openwrt.git WhatEverDirectoryName (downloads the 19.07 branch of OpenWRT, which also includes the latest version, 19.07.3)
  • To see repository, in web browser: https://git.openwrt.org/openwrt/
  • git tag (essentially shows available versions to build)
  • git branch (shows local versions of OpenWRT downloaded)
  • git branch -r (shows versions available for download)
• Feeds
  • Documentation: https://openwrt.org/docs/guide-developer/feeds
  • ~/openwrt/feeds.conf.default
  • ./scripts/feeds update -a (A script from OpenWRT to makes sure all files are up to date)
  • ./scripts/feeds (by itself, no "sub-commands", shows available options)
  • ./scripts/feeds list -n (shows main categories of feeds)
    • Should show these Names: packages, luci, routing, and telephony
  • ./scripts/feeds list -r OneOfTheFeedNames (shows the available packages, names are equivalent to what would be found with opkg list)
  • Example: ./scripts/feeds install htop (an enhanced version of top)
    • Files will be downloaded here: ~/openwrt/package/feeds/packages/, which are symbolic links to: ~/openwrt/feeds/packages/admin
• Checkout (not needed if a single branch like 19.07 is downloaded)
  • git checkout openwrt-19.07 (if multiple branches have been downloaded, switch to using this one)
  • Oddly, this command is documented using the latest version of OpenWRT as of 9.2020, but it doesn't work as expected: git checkout v19.07.03 (from https://openwrt.org/docs/guide-developer/build-system/use-buildsystem)
• SDK
  • Download the SDK and uncompress it (which is equivalent to the git clone command above)
    • URL for SDK (look towards the bottom of the page for SDK): https://downloads.openwrt.org/releases/19.07.3/targets/mvebu/cortexa9/
    • Don't worry about the name ending in x86_64 as this indicates the platform that the cross compiling will take place on
  • tar xf WhatEverFileName (be sure one is in the ~ directory)
  • OR... Do it manually (based on instructions from: https://openwrt.org/docs/guide-developer/helloworld/chapter1);
    • git clone -b v19.07.3 https://github.com/openwrt/openwrt.git ~/openwrtSDK (NOTE: The instructions at the above address do not provide the correct URL when cloning)
    • git checkout v19.07.3 (If a specific version is specified there is no need to "checkout" a version as specified in the same instructions, plus it appears since the instructions have been written there are other naming conventions too, like openwrt-19.07, but choosing a specific version is better.)
    • make distclean wipes out all configuration, starts from a clean slate
    • ./scripts/feeds update -a: Downloads pa
  • ./scripts/feeds update -a
  • ./scripts/feeds install WhatEverPackageName
  • ToolChain Directory: ~/openwrt/staging_dir/
    • Linux ToolChains & Cross Compiling: https://elinux.org/Toolchains
    • ToolChain: https://en.wikipedia.org/wiki/Toolchain
    • Cross Compiler: https://en.wikipedia.org/wiki/Cross_compiler
    • musl: https://www.musl-libc.org/intro.html
    • EABI (Embedded Application Binary Interface): https://en.wikipedia.org/wiki/Application_binary_interface
• Package Building
  • Sources
    • https://openwrt.org/docs/guide-developer/package-policies
    • https://openwrt.org/docs/guide-developer/packages
    • https://openwrt.org/docs/guide-developer/helloworld/chapter3
  • Using the HTOP package as an example
    • Make a Manifest: https://openwrt.org/docs/guide-developer/helloworld/chapter3 (can copy similar to the below)
    • Find a "template" (similar existing package) and copy it: cp -R ~/openwrt/package/feeds/packages/htop ~/openwrt/package/feeds/packages/htopcustom (it's a symbolic link, so cp -R), modify and make changes
    • make menuconfig
      • PKG_NAME and PKG_VERSION need to change at minimum to get it to show up as a new option
    • disable all the Global settings to "build all"
    • Utilities (look in the Makefile file in the define Package/tarcustom (or WhatEverName) for where the package will be)
• Headers
  • Potential Locations;
    • ~/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/linux-4.14.195/user_headers/include
  • Sources
    • https://forum.openwrt.org/t/how-to-obtain-headers-for-openwrts-kernel/50737/6
    • https://openwrt.org/docs/guide-developer/packages

Environment Variables

export FORCE_UNSAFE_CONFIGURE=1

Standard Location: PATH=~/openwrt/staging_dir/host/bin:$PATH

For examples on OpenWRT "Hello World" Examples: export PATH=/home/buildbot/source/staging_dir/host/bin:$PATH

For persistent settings across reboots or logins: Add to ~/.bash_profile file

feeds.conf.default, feeds.conf, ~/openwrt/feeds.con, ~/openwrt/

Script that attempts to set Linux environment default variables for Make / Cmake: ~/openwrt/staging_dir/host/share/cmake-3.15/Modules/Platform/UnixPaths.cmake

Other package default path variables ~/openwrt/include/package-defaults.mk

• Created when git clone is used
  • Feeds Directory: ~/openwrt/feeds/packages/admin, utils, etc.
  • Feeds Symbolic Links in this Directory (for convenience?): ~/openwrt/package/feeds/packages/ (created when the ./scripts/feeds update -a command is used)

Notes on 9.7.2020

On Monday 9.7.2020 during the late afternoon (UTC-8) it was observed that OpenWRT 19.07.4 was released on Tuesday 9.8.2020. This would seem to indicate the server for OpenWRT downloads is somewhere in Europe.

• Downloads.OpenWRT.org: 176.9.48.73 is allocated to a company located in Germany (https://search.arin.net/rdap/?query=176.9.48.73)
• www.OpenWRT.org: 139.59.209.225 is allocated to a company in New York, United States (https://search.arin.net/rdap/?query=139.59.209.225)

The following interesting observations were made after the above noted event

• git tag : Lists all "major" and "minor" version branches of OpenWRT available.
  • v17.01.0
  • v17.01.0-rc1
  • v17.01.0-rc2
  • v17.01.1
  • v17.01.2
  • v17.01.3
  • v17.01.4
  • v17.01.5
  • v17.01.6
  • v17.01.7
  • v18.06.0
  • v18.06.0-rc1
  • v18.06.0-rc2
  • v18.06.1
  • v18.06.2
  • v18.06.3
  • v18.06.4
  • v18.06.5
  • v18.06.6
  • v18.06.7
  • v18.06.8
  • v19.07.0
  • v19.07.0-rc1
  • v19.07.0-rc2
  • v19.07.1
  • v19.07.2
  • v19.07.3
  • v19.07.4

• git branch -r : Shows available "major versions" of OpenWRT available.
  •   origin/HEAD -> origin/master
  •   origin/lede-17.01
  •   origin/master
  •   origin/openwrt-18.06
  •   origin/openwrt-19.07

• git clone -b v19.07.3 https://github.com/openwrt/openwrt.git produces the following output
  • Cloning into '/root/openwrtSDK'... remote: Enumerating objects: 13, done. remote: Counting objects: 100% (13/13), done. remote: Compressing objects: 100% (11/11), done. remote: Total 530753 (delta 0), reused 6 (delta 0), pack-reused 530740 Receiving objects: 100% (530753/530753), 183.81 MiB | 3.68 MiB/s, done. Resolving deltas: 100% (362276/362276), done. Note: checking out 'f3f38f40da74a193b4177be8d0fa746d00a753ea'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b new_branch_name
  • ...and git branch produces this output: fatal: Not a git repository (or any of the parent directories): .git. (Given that "HEAD" refers to a local copy of OpenWRT, this is not an issue)
• git clone -b openwrt-19.07 https://github.com/openwrt/openwrt.git produces the following output
  • Cloning into '/root/openwrt-19.07'... remote: Enumerating objects: 13, done. remote: Counting objects: 100% (13/13), done. remote: Compressing objects: 100% (11/11), done. remote: Total 530753 (delta 0), reused 6 (delta 0), pack-reused 530740 Receiving objects: 100% (530753/530753), 183.81 MiB | 3.69 MiB/s, done. Resolving deltas: 100% (362276/362276), done.
  • ...and git branch produces this output: * openwrt-19.07 (the asterisk indicates the "checked out" version if multiple versions existed)
• On 9.6.2020 this was contents of the feeds file included with cloning OpenWRT using openwrt-19.07;
  • src-git base https://git.openwrt.org/openwrt/openwrt.git;v19.07.3
  • src-git packages https://git.openwrt.org/feed/packages.git^2e6bd4cb86682b224803325127d3f777d40b3231
  • src-git luci https://git.openwrt.org/project/luci.git^fb2f36306756d0d0782dcab8413a8bb7ec379e54
  • src-git routing https://git.openwrt.org/feed/routing.git^3f8571194c2765ed31aa73459e86c2ebf943d27d
  • src-git telephony https://git.openwrt.org/feed/telephony.git^036cd451c35b82b3d8cac519864986894d9f6958

• On 9.7.2020 this was contents of the feeds file included with cloning OpenWRT with both the openwrt-19.07 and v19.07.3;
  • src-git packages https://git.openwrt.org/feed/packages.git;openwrt-19.07
  • src-git luci https://git.openwrt.org/project/luci.git;openwrt-19.07
  • src-git routing https://git.openwrt.org/feed/routing.git;openwrt-19.07
  • src-git telephony https://git.openwrt.org/feed/telephony.git;openwrt-19.07