OpenWRT Cross Compiling

• CentOS 7.8, OpenWRT 19.07.3, Perl 5.28.1
  • ...a working CentOS system.
  • yum install binutils bzip2 gcc gcc-c++ gawk gettext flex ncurses-devel zlib-devel zlib-static make patch unzip openssl-devel perl-ExtUtils-MakeMaker glibc glibc-devel glibc-static ncurses-libs sed sdcc intltool sharutils bison wget git-core xz
  • OpenWRT, browse GIT Repository: https://git.openwrt.org/
  • git clone -b openwrt-19.07 https://github.com/openwrt/openwrt.git (this should download just the 19.07.01, .03, .04, etc branch)
• Directory: The default for most of the items below is the users "home" directory ( ~ )
• Perlbrew for matching the exact version of Perl on the router and controlling it easily: https://perlbrew.pl/
  • Nice tutorial for Perlbrew basics: https://www.digitalocean.com/community/tutorials/how-to-install-perlbrew-and-manage-multiple-versions-of-perl-5-on-centos-7
  • perlbrew list
  • perlbrew list-modules
  • perlbrew use NameOfVersion
  • perl -v: Verify current version used
  • Install a Perl Module: curl -L https://cpanmin.us | perl - Name::Name
• Download pre-requisites for a Cross Compile Build system: https://openwrt.org/docs/guide-developer/build-system/install-buildsystem
  • EPEL repository needed for some: yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
  • yum install binutils bzip2 gcc gcc-c++ gawk gettext flex ncurses-devel zlib-devel zlib-static make patch unzip perl-ExtUtils-MakeMaker glibc glibc-devel glibc-static ncurses-libs sed sdcc intltool sharutils bison wget git-core openssl-devel xz
    • And of course some of the above (zlib-static, glibc-static, sdcc, shareutils) doesn't work and is also a bit ambivalent, so;
      • yum install zlib glibc-common glibc-devel glibc-headers glibc-langpack-en (sdcc (compiler for 8501 microcontroller) and shareutils (for specific encoding for paid RedHat subscribers) are not readily available for CentOS 8)
• If one wants to build firmware version of OpenWRT: Download OpenWRT Files (from https://openwrt.org/docs/guide-developer/build-system/install-buildsystem)
  • git clone https://git.openwrt.org/openwrt/openwrt.git
• If one wants to build Perl and compile modules that include other libraries such as OpenSSL
  • yum install openssl-devel
  • Download the TAR package from CPAN
  • perl Makefile.PL
  • make
  • make test
  • make dist
• OpenSSL Cross Compiling (make sure the appropriate version of Perl is selected via perlbrew, above)
  • Links;
    • https://stackoverflow.com/questions/15539062/cross-compiling-of-openssl-for-linux-arm-v5te-linux-gnueabi-toolchain
    • https://wiki.openssl.org/index.php/Compilation_and_Installation
    • For ARM infor and --prefix and --openssldir info: https://stackoverflow.com/questions/15539062/cross-compiling-of-openssl-for-linux-arm-v5te-linux-gnueabi-toolchain
    • https://centos.pkgs.org/7/epel-x86_64/binutils-arm-linux-gnu-2.27-9.el7.1.x86_64.rpm.html
    • https://pkgs.org/search/?q=binutils-arm-linux-gnu
  • wget OpenSSL URL ( https://www.openssl.org/source/ )
  • ./Configure LIST ( linux-armv4 )
  • ./Configure linux-armv4 ( ./config is equivalent to ./Configure x86_64 )
  • --prefix=/WhatEverPathForCompiledFiles/
  • --openssldir=/usr/ (that's where OpenWRT puts openssl, /usr/bin/ )
  • make
  • make depend
  • make dist, err, make tar (for OpenSSL, but it doesn't work of course);
    • Key piece of information for doing it manually: https://stackoverflow.com/questions/1506462/what-files-did-make-install-copy-and-where and https://stackoverflow.com/questions/11307465/destdir-and-prefix-of-make
    • make install DESTDIR=
  • Header files should be in /usr/include
• Compiling
  • file NameOfBinary: Shows information about binary file (opkg install file)
  • Shared VS Non-Shared, Static VS Shared, .a VS .so ( using the shared switch, as in ./Configure XYZ shared
    • Static: EVERYTHING in a single binary / executable file
    • Shared: Dependencies located in other files

...quitting for now because the instructions at https://openwrt.org/docs/guide-developer/build-system/use-buildsystem#selecting_a_specific_major_revision are a pathetic joke. Go ahead, try them, see if it works. Nope....

Additional notes from other failed attempts;

http://transit.iut2.upmf-grenoble.fr/cgi-bin/man/man2html?Crypt::SSLeay+3

https://github.com/Entware/Entware-ng/wiki/Using-gcc-%28native-compilation%29

perl Makefile.PL --incpath /usr/src/lib

perl Makefile.PL -- $openssl_path=/usr/src/lib/openssl

echo | gcc -v -E -

perl -V

/usr/src/net/Net_SSLeay.pm-1.23/MYMETA.yml

env

opkg install perl-dev

opkg install openssl-

https://github.com/Entware/Entware-ng/wiki/Self-installation-of-perl-modules

https://github.com/Entware/Entware-ng/issues/430

https://github.com/Entware/Entware/wiki/Using-GCC-for-native-compilation

/usr/bin/ld: cannot find -lssl

/usr/bin/ld: cannot find -lcrypto

/usr/bin/ld: cannot find -ldl

ln -s

ld -lcrypto --verbose

find /

https://stackoverflow.com/questions/16710047/usr-bin-ld-cannot-find-lnameofthelibrary

For the /usr/bin/ld Issue: https://stackoverflow.com/questions/16710047/usr-bin-ld-cannot-find-lnameofthelibrary

/usr/bin/opensslOpenSSL 1.1.1g  21 Apr 2020

built on: Thu Jul 30 19:43:27 2020 UTC

platform: linux-arm-openwrt

options:  bn(64,32) rc4(char) des(long) blowfish(ptr)

compiler: arm-openwrt-linux-muslgnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O3 -Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -mfloat-abi=hard -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -fpic -ffunction-sections -fdata-sections -znow -zrelro -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DNDEBUG -DOPENSSL_PREFER_CHACHA_OVER_GCM -DOPENSSL_SMALL_FOOTPRINT

OPENSSLDIR: "/etc/ssl"

ENGINESDIR: "/usr/lib/engines-1.1"

Seeding source: os-specific

OpenSSL>version -a

https://www.cpan.org/modules/by-module/Net/Net-SSLeay-1.73.readme

https://stackoverflow.com/questions/60521205/how-can-i-install-netssleay-with-perlbrew-in-macos-catalina

perl Makefile.PL -- INC=-I/usr/src/lib/ -- OPENSSL_PREFIX=/usr/src/openssl_compiled/usr/local

SSLeay.c: In function 'XS_Net__SSLeay_CTX_sessions':

SSLeay.c:3608:2: error: unknown type name 'LHASH'; did you mean '_LHASH'?

  LHASH * RETVAL;

  ^~~~~

 

  SSLeay.c line 3608, change LHASH to _LHASH

OpenSSL

remove -ldl and all -pthread references near EX_LIBS (from https://github.com/openssl/openssl/issues/3106)

Change destination directory for GCC Make: https://stackoverflow.com/questions/3239343/make-install-but-not-to-default-directories

...sure enough, in Makefile;

# DESTDIR is for package builders so that they can configure for, say,

# /usr/ and yet have everything installed to /tmp/somedir/usr/.

# Normally it is left empty.

DESTDIR=

Add the paths in Make File that include the new SSL stuff

DDLFLAGS = -shared -L/usr/src/openssl_compiled/usr/local/ -L/usr/src/openssl_compiled/usr/local/lib -L/usr -L/usr/lib -L/builder/shared-workdir/build/sdk/staging_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/usr/lib -L/builder/shared-workdir/build/sdk/staging_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/lib -L/builder/shared-workdir/build/sdk/staging_dir/toolchain-arm_cortex-a9+vfpv3-d16_gcc-7.5.0_musl_eabi/usr/lib -L/builder/shared-workdir/build/sdk/staging_dir/toolchain-arm_cortex-a9+vfpv3-d16_gcc-7.5.0_musl_eabi/lib -znow -zrelro

LDFLAGS = -L/usr/src/openssl_compiled/usr/local/ -L/usr/src/openssl_compiled/usr/local/lib -L/usr -L/usr/lib -L/builder/shared-workdir/build/sdk/staging_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/usr/lib -L/builder/shared-workdir/build/sdk/staging_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/lib -L/builder/shared-workdir/build/sdk/staging_dir/toolchain-arm_cortex-a9+vfpv3-d16_gcc-7.5.0_musl_eabi/usr/lib -L/builder/shared-workdir/build/sdk/staging_dir/toolchain-arm_cortex-a9+vfpv3-d16_gcc-7.5.0_musl_eabi/lib -znow -zrelro