Compiling on OpenWRT

These are some notes, we'll see if it works and warrants a full article...

• Entware has better resources documentation for compiling on a router: https://github.com/Entware/Entware/wiki/Using-GCC-for-native-compilation
• A key resource are headers;
  • To keep OpenWRT and Entware stuff separate, create the /opt directory: mkdir /opt
  • Then it is safe to install / download this: wget -qO- http://bin.entware.net/armv7sf-k3.2/include/include.tar.gz | tar xvz -C /opt/include
• And from this source, it is easy to add the path when compiling: https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html
  • By default, gcc searches these directories first: /usr/include and /usr/lib/gcc/arm-openwrt-linux-muslgnueabi/7.4.0/include
  • Downloading the above Entware Header File, this command can be used to search a different directory, like /opt/include, first: -IWhatEverDirectory
• Items to install;
  • GCC:
    • opkg install gcc
  • Make: https://man7.org/linux/man-pages/man1/make.1.html
    • opkg install make
  • Entware recommends installing the following;
    • opkg install busybox ldd make gawk sed (ldd and sed are not available as separate packages in OpenWRT, but both are included within the OpenWRT full version of BusyBox)
  • Additional utilities are recommended
    • Install the full version of BASH: opkg install bash
• Environment Variables to add (these can be added in a myriad of ways, see the end of this section;
  • export LDFLAGS="-Wl,-rpath=/usr/lib:/opt/lib -Wl,--dynamic-linker=/usr/lib/gcc/arm-openwrt-linux-muslgnueabi/7.4.0/ld-musl-armhf.so.1 -L/usr/lib:/opt/lib:/opt/include
  • NOTE: The original DD-WRT / Entware version is this: export LDFLAGS="-Wl,-rpath=/opt/lib -Wl,--dynamic-linker=/opt/lib/ld-linux.so.3 -L/opt/lib", and needs to be modified for OpenWRT as shown above.
    • -WI is a switch that allows for additional items to be added via the GNU LD (https://stackoverflow.com/questions/6562403/i-dont-understand-wl-rpath-wl) command
    • -rpath in LDFLAGS = "-rpath=/usr/lib:/opt/lib" is the same as executing this on the command line: export LD_LIBRARY_PATH = "/opt/lib
      • rpath AND/OR LD_LIBRARY_PATH is a path used by an executable program to search directories containing shared libraries after the program has been compiled (from https://en.wikipedia.org/wiki/Rpath and https://stackoverflow.com/questions/4250624/ld-library-path-vs-library-path)
      • OpenWRT has its own equivalent path, which is /usr/lib, however there are some variations in files and sizes. Solution? Include both paths (from https://www.linuxtopia.org/online_books/an_introduction_to_gcc/gccintro_24.html);
        • -rpath=/usr/lib:/opt/lib (this will search OpenWRT's /usr/lib directory first, then DD-WRT / Entware's /opt/lib directory second)
    • --dynamic-linker
      • For DD-WRT / Entware, the GNU LD file is located here: /opt/lib/ld-linux.so.3, which is a symbolic link to ld-2.27.so (and here too: /lib/gcc/arm-openwrt-linux-gnueabi/7.4.0)
      • OpenWRT seems to have an equivalent GNU LD file located here: /usr/lib/gcc/arm-openwrt-linux-muslgnueabi/7.4.0/ld-musl-armhf.so.1, so use that path instead (from https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/file-not-found-when-executing-assembled-program)
        • --dynamic-linker /usr/lib/gcc/arm-openwrt-linux-muslgnueabi/7.4.0/ld-musl-armhf.so.1
      • LD is not the same as GNU LD (https://en.wikipedia.org/wiki/Linker_(computing)#GNU_linker)
    • -L in LDFLAGS = "-L/usr/lib:/opt/lib:/opt/include" is the same as executing this on the command line: export LIBRARY_PATH=/usr/lib:/opt/lib:/opt/include (https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html)
      • LIBRARY_PATH is used by gcc before compilation to search directories containing static and shared libraries that need to be linked to your program (from https://stackoverflow.com/questions/4250624/ld-library-path-vs-library-path)
  • export CFLAGS="-O2 -pipe -march=armv7-a -mtune=cortex-a9 -fno-caller-saves -mfloat-abi=soft -I/opt/include "
  • NOTE: The original DD-WRT / Entware version is this: export CFLAGS="-O2 -pipe -march=armv7-a -mtune=cortex-a9 -fno-caller-saves -mfloat-abi=soft ", and needs to be modified for OpenWRT as shown above.
    • -I in CFLAGS = "-I/opt/include" is the same as: export CPATH=/opt/include (from https://gcc.gnu.org/onlinedocs/cpp/Environment-Variables.html)
    • NOTE: that's an "I" (eye) in the above command, as in the word "Inside, Inner, Implode, etc., not a "l" (el), as in the word Little, Lime, etc.
  • Adding the above Environment Variables;
    • Run each of the below command lines
      • export LDFLAGS="-Wl,-rpath=/usr/lib:/opt/lib -Wl,--dynamic-linker=/usr/lib/gcc/arm-openwrt-linux-muslgnueabi/7.4.0/ld-musl-armhf.so.1 -L/usr/lib:/opt/lib:/opt/include
      • export CFLAGS="-O2 -pipe -march=armv7-a -mtune=cortex-a9 -fno-caller-saves -mfloat-abi=soft -I/opt/include "
    • Put the above two EXPORT commands in a single file, and run the following command;
      • source WhatEverFileName
    • Add the above two EXPORT commands to a user's profile
      • nano ~/.profile
      • NOTE: For the above to work, the full BASH shell must be installed (opkg install bash) AND the ~/.profile file must include the directive: SHELL=/bin/bash
• ...as far as the Entware documentation for Compiling: https://github.com/Entware/Entware/wiki/Using-GCC-for-native-compilation
  • Additional information for the ./configure --prefix /opt command, and what --prefix means: https://askubuntu.com/questions/891835/what-does-prefix-do-exactly-when-used-in-configure
  • Although they offer a link and a command for downloading additional libraries to /opt/include, there doesn't appear to be any notation of where that is utilized. Just having it as a bunch of files in a directory is useless unless a compiler like GCC can make use of the files. It should be
• ELF Definition: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format

Example

• Create a file named ex.sh: nano ex.sh
• Add the following directives (from the above section) and save it;

export LDFLAGS="-Wl,-rpath=/usr/lib:/opt/lib -Wl,--dynamic-linker=/usr/lib/gcc/arm-openwrt-linux-muslgnueabi/7.4.0/ld-musl-armhf.so.1 -L/usr/lib:/opt/lib:/opt/include

export CFLAGS="-O2 -pipe -march=armv7-a -mtune=cortex-a9 -fno-caller-saves -mfloat-abi=soft -I/opt/include "

• Create a "Hello World" source code file: nano hw.c
• Add the following text and save it;

#include <stdio.h>    

int main(void)    
{
printf("\nHello, world!\n\n");        
return 0;        
}

The below two commands are from an OpenWRT example here: https://openwrt.org/docs/guide-developer/helloworld/chapter2

• Type this command: gcc -c -o hw.o hw.c -Wall
• Type this command: gcc -o hw hw.o

...and then run it: ./hw, which should produce the below output;

Hello, world!