new build system for fxSDK sysroot and libstdc++-v3

The main change is making the scripts a two-stage process where we
first build GCC, leave the user to install the libc, and then come back
to install libstdc++-v3.

* Detect whether we are in the first or second stage
* Don't clean files after first stage install, and even then only do it
  if :clean is specified
* Update README except for the manual install tutorial
* Patch the source using a backported GCC 12.1 commit to make the
  configure script for libstdc++-v3 skip checking for dlopen, which is
  impossible is our not-really-hosted setup

Details:
* Proper .gitignore

.gitignore

@@ -1,12 +1,4 @@
-# Everything but the scripts
+/gcc-*/
-*
+/gcc-*.tar.*
-!giteapc.make
+/build*/
-!giteapc-config-any.make
+/giteapc-config.make
-!giteapc-config-noclean.make
-!configure.sh
-!build.sh
-!install.sh
-!uninstall.sh
-!util.sh
-!.gitignore
-!README.md

README.md

@@ -1,24 +1,54 @@
-# Automatic `sh-elf-gcc` installer
+# SuperH toolchain: `sh-elf-gcc`
 
-This script can be used to automatically compile and install a GCC cross-compiler targeting SH3 and SH4 calculators. The normal use is with GiteaPC:
+This repository provides scripts to automatically compile and install an SH3/SH4-compatible [GCC cross-compiler](https://gcc.gnu.org/). GCC is a collection of compilers most commonly used for C/C++.
 
-```
+The following three methods can be used to install the compiler with different levels of automation.
+
+Note that this repository should usually be built twice: first to build the compiler, and then after the libc is installed to build the C++ library.
+
+## Method 1: Using GiteaPC
+
+The most common way to install this compiler is for the fxSDK, and it can be automated with [GiteaPC](/Lephenixnoir/GiteaPC):
+
+```bash
 % giteapc install Lephenixnoir/sh-elf-gcc
 ```
 
-You can also install manually. First install [`sh-elf-binutils`](https://gitea.planet-casio.com/Lephenixnoir/sh-elf-binutils), then run the GiteaPC Makefile with a manually-specified install prefix:
+This installs GCC (and binutils if missing) in the fxSDK's SuperH system root. Note that at first it will *not* install the C++ standard library libstdc++, because it requires the C standard library which is not available at this stage. After you install [fxlibc](/Vhex-Kernel-Core/fxlibc/) you should run GiteaPC's install command again, and this time the scripts will build libstdc++. The GiteaPC tutorial has more detailed instructions about this two-stage process.
 
-```
+A `:any` configuration is provided in case you already have another version of GCC installed in the fxSDK sysroot and want to keep using it (ie. skip a version upgrade). This will mark this repository as installed, so other repositories depending on it can build, without actually compiling binutils.
-% make -f giteapc.make configure build install PREFIX=$HOME/.local
-```
 
-An `any` configuration is provided in case GCC is already installed externally, to have this package installed without rebuilding it.
+```bash
-
-```
 % giteapc install Lephenixnoir/sh-elf-gcc:any
 ```
 
-## Notes on building libstdc++-v3
+A `:clean` configuration is also provided if you want to clean up the source and build files automatically after the second pass. This frees up some disk space.
+
+```bash
+% giteapc install Lephenixnoir/sh-elf-gcc:clean
+```
+
+## Method 2: Manually running the scripts
+
+Make sure to previously install:
+
+* [fxSDK](https://gitea.planet-casio.com/Lephenixnoir/fxsdk) ≥ 2.8.1 (provides the sysroot)
+* [binutils for SuperH](https://gitea.planet-casio.com/Lephenixnoir/sh-elf-binutils)
+* Most of the [requirements for GCC](https://gcc.gnu.org/install/prerequisites.html) are checked in the binutils `configure.sh`
+
+Follow the same procedure as for binutils; preferably use the same `PREFIX`.
+
+```bash
+% make -f giteapc.make configure build install PREFIX="$HOME/.local"
+```
+
+## Method 3: Fully manually
+
+TODO: Manual install tutorial and link to the Planète Casio one
+
+TODO: The stuff below about libstdc++ is outdated, we can now build the entire thing
+
+**Notes on building libstdc++-v3**
 
 These are experimental notes on attempts at building the C++ standard library implementation bundled with GCC, `libstdc++-v3`. For the official manual, see [libstdc++ info manual, Chapter 2: Setup](https://gcc.gnu.org/onlinedocs/libstdc++/manual/setup.html) (gcc.gnu.org).
 
@@ -81,17 +111,3 @@ If it succeeds, install.
 ```
 % make install-strip-target-libstdc++-v3
 ```
-
-### Current problems
-
-Hard problems:
-
-* None. The free-standing subset compiles.
-
-Things that look like they could be involved in problems:
-
-* Anything that is not in the fxlibc can fail to link.
-
-* The conftest programs are built without `-ffreestanding`, which means autoconf cannot really link stuff. This is probably not too much of a problem, because it's cross-compiled anyway so there's nothing to do with a linked program, but who knows.
-
-* autoconf compiles conftest programs as if on a fully-featured dynamic OS, which is nowhere near true (eg. it builds with `-shared-libgcc`).

build.sh

@@ -1,7 +1,7 @@
 #! /usr/bin/env bash
 
 # Avoid rebuilds of the same version
-[[ ! -d build ]] && exit 0
+[[ -e "build/giteapc-skip-rebuild.txt" ]] && exit 0
 
 source util.sh
 cd build
@@ -13,12 +13,12 @@ else
   cores=$(nproc)
 fi
 
-if command -v gmake >/dev/null 2>&1; then
+if [[ -e "giteapc-build-libstdcxx.txt" ]]; then
-  make_command=gmake
+  echo "$TAG Compiling libstdc++-v3..."
+  run_quietly giteapc-build-libstdcxx.log \
+  $MAKE_COMMAND -j"$cores" all-target-libstdc++-v3
 else
-  make_command=make
+  echo "$TAG Compiling gcc (usually 10-20 minutes)..."
+  run_quietly giteapc-build.log \
+  $MAKE_COMMAND -j"$cores" all-gcc all-target-libgcc
 fi
-
-echo "$TAG Compiling gcc (usually 10-20 minutes)..."
-run_quietly giteapc-build.log \
-$make_command -j"$cores" all-gcc all-target-libgcc

configure.sh

@@ -7,34 +7,70 @@ PREFIX="$2"
 URL="https://ftp.gnu.org/gnu/gcc/gcc-$VERSION/gcc-$VERSION.tar.xz"
 ARCHIVE=$(basename "$URL")
 
-# Avoid rebuilds of the same version
+# Final location of gcc in the sysroot
+SYSROOT_GCC="$SYSROOT/bin/sh-elf-gcc"
+# Version string of any existing sh-elf-gcc in the sysroot
+SYSROOT_GCC_VERSION=
+# "1" if libstdc++-v3 is already in the install folder
+SYSROOT_HAS_LIBSTDCXX=
+# "1" if the conditions for building libstdc++-v3 are met
+CAN_BUILD_LIBSTDCXX=
 
-existing_gcc="$PREFIX/bin/sh-elf-gcc"
+#---
+# Determine what parts of GCC are already installed and decide whether to build
+#---
 
-if [[ -f "$existing_gcc" ]]; then
+if [[ -f "$SYSROOT_GCC" ]]; then
-  existing_version=$(sh-elf-gcc --version | head -n 1 | grep -Eo '[0-9.]+$')
+  SYSROOT_GCC_VERSION=$("$SYSROOT_GCC" --version | head -n 1 | grep -Eo '[0-9.]+$')
-  if [[ $existing_version == $VERSION ]]; then
+
-    echo "$TAG Version $VERSION already installed, skipping rebuild"
+  # TODO: Once we start using the full C++ library, check for libstdc++.a instead
-    if [[ -e build ]]; then
+  LIBSTDCXX_FILE=$("$SYSROOT_GCC" -print-file-name=libsupc++.a)
-      rm -rf build
+  LIBC_FILE=$("$SYSROOT_GCC" -print-file-name=libc.a)
+
+  if [[ "$LIBSTDCXX_FILE" != "libsupc++.a" ]]; then
+    SYSROOT_HAS_LIBSTDCXX=1
+    echo "$TAG libsupc++.a found, libstdc++-v3 is already built"
   fi
+
+  if [[ "$LIBC_FILE" != "libc.a" ]]; then
+    CAN_BUILD_LIBSTDCXX=1
+    [[ "$SYSROOT_HAS_LIBSTDCXX" != "1" ]] && \
+      echo "$TAG libc.a found, we can build libstdc++-v3"
+  else
+    echo "$TAG libc.a not found, we cannot build libstdc++-v3"
+  fi
+else
+  echo "$TAG After installing the libc, rebuild here for libstdc++-v3"
+fi
+
+if [[ "$SYSROOT_GCC_VERSION" = "$VERSION" ]]; then
+  if [[ "$SYSROOT_HAS_LIBSTDCXX" = "1" ]]; then
+    echo "$TAG GCC $VERSION with libstdc++-v3 already there; skipping rebuild"
+    mkdir -p build
+    touch build/giteapc-skip-rebuild.txt
     exit 0
+  elif [[ "$CAN_BUILD_LIBSTDCXX" != "1" ]]; then
+    echo "$TAG Cannot build libstdc++-v3 yet; skipping rebuild until next time"
+    mkdir -p build
+    touch build/giteapc-skip-rebuild.txt
+    exit 0
+  else
+    echo "$TAG We will proceed with the second stage and build libstdc++-v3"
   fi
 fi
 
-# Aggressive parameter to avoid rebuilds
+if [[ ! -z "$ACCEPT_ANY" && ! -z "$SYSROOT_GCC_VERSION" ]]; then
-
+  echo "$TAG GCC $VERSION already available; skipping rebuild as per ACCEPT_ANY"
-if [[ ! -z "$ACCEPT_ANY" ]]; then
+  mkdir -p build
-  if command -v sh-elf-gcc >/dev/null 2>&1; then
+  touch build/giteapc-skip-rebuild.txt
-    echo "$TAG sh-elf-gcc in PATH and ACCEPT_ANY is set, skipping build"
-    if [[ -e build ]]; then
-      rm -rf build
-    fi
   exit 0
-  fi
 fi
 
-# Download archive
+[[ -e "build/giteapc-skip-rebuild.txt" ]] && rm build/giteapc-skip-rebuild.txt
+
+#---
+# Get sources
+#---
 
 if [[ -f "$ARCHIVE" ]]; then
   echo "$TAG Found $ARCHIVE, skipping download"
@@ -50,44 +86,76 @@ else
   fi
 fi
 
-# Extract archive (openBSD-compliant version)
+# Extract archive (OpenBSD-compliant version)
+if [[ -d "${ARCHIVE%.tar.*}" ]]; then
+  echo "$TAG Found ${ARCHIVE%.tar.*}, skipping extraction"
+else
+  echo "$TAG Extracting $ARCHIVE..."
+  unxz -c < $ARCHIVE | tar -xf -
 
-echo "$TAG Extracting $ARCHIVE..."
+  if [[ "$VERSION" = "11.1.0" ]]; then
-unxz -c < $ARCHIVE | tar -xf -
+    echo "$TAG Applying patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch..."
+    patch -u -N -p0 < patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch
+  fi
+fi
 
 # Download prerequisites
-
 cd gcc-$VERSION
 ./contrib/download_prerequisites
 cd ..
 
-# Create build folder
+#---
+# Configure
+#---
 
-[[ -d "build" ]] && rm -rf build
+# We install in the sysroot and then symlink to the PREFIX folder in the path.
-mkdir build
-
-# Configure. GCC does not support make uninstall so we install in this
-# directory and later symlink executables to $PREFIX/bin.
-
-PREFIX="$(pwd)"
 
 # Symlink as, ld, ar and ranlib, which gcc will not find by itself (we renamed
 # them from sh3eb-elf-* to sh-elf-* with --program-prefix).
-mkdir -p sh3eb-elf/bin
+mkdir -p "$SYSROOT/sh3eb-elf/bin"
-ln -sf $(command -v sh-elf-as) sh3eb-elf/bin/as
+for TOOL in as ld ar ranlib; do
-ln -sf $(command -v sh-elf-ld) sh3eb-elf/bin/ld
+  if ! command -v sh-elf-$TOOL >/dev/null 2>&1; then
-ln -sf $(command -v sh-elf-ar) sh3eb-elf/bin/ar
+    echo "error: sh-elf-$TOOL not found in PATH!" >&2
-ln -sf $(command -v sh-elf-ranlib) sh3eb-elf/bin/ranlib
+    exit 1
-
+  fi
-cd build
+  ln -sf $(command -v sh-elf-$TOOL) "$SYSROOT/sh3eb-elf/bin/$TOOL"
+done
 
 # OpenBSD apparently installs these in /usr/local
 if [[ $(uname) == "OpenBSD" ]]; then
-  extra_args="--with-gmp=/usr/local --with-mpfr=/usr/local --with-mpc=/usr/local"
+  EXTRA_ARGS="--with-gmp=/usr/local --with-mpfr=/usr/local --with-mpc=/usr/local"
 else
-  extra_args=
+  EXTRA_ARGS=
 fi
 
-echo "$TAG Configuring gcc..."
+# In libstdc++-v3 mode, don't reconfigure to avoid recompiling
-run_quietly giteapc-configure.log \
+if [[ "$SYSROOT_GCC_VERSION" = "$VERSION" && "$SYSROOT_HAS_LIBSTDCXX" != "1" && "$CAN_BUILD_LIBSTDCXX" = "1" ]]; then
-../gcc-$VERSION/configure --prefix="$PREFIX" --target=sh3eb-elf --with-multilib-list=m3,m4-nofpu --enable-languages=c,c++ --without-headers --with-newlib --program-prefix=sh-elf- --enable-libssp --enable-lto $extra_args
+  touch build/giteapc-build-libstdcxx.txt
+elif [[ "$SYSROOT_GCC_VERSION" != "$VERSION" ]]; then
+  echo "$TAG Configuring gcc..."
+  [[ -d build ]] && rm -rf build
+  mkdir build
+
+  cd build
+  run_quietly giteapc-configure.log         \
+  ../gcc-$VERSION/configure                 \
+    --prefix="$SYSROOT"                     \
+    --target="sh3eb-elf"                    \
+    --with-multilib-list="m3,m4-nofpu"      \
+    --enable-languages="c,c++"              \
+    --without-headers                       \
+    --program-prefix="sh-elf-"              \
+    --enable-libssp                         \
+    --enable-lto                            \
+    --enable-clocale="generic"              \
+    --enable-libstdcxx-allocator            \
+    --disable-threads                       \
+    --disable-libstdcxx-verbose             \
+    --enable-cxx-flags="-fno-exceptions"    \
+    $EXTRA_ARGS
+  cd ..
+else
+  echo "$TAG GCC already built, can't build libstdc++-v3; skipping!"
+  touch build/giteapc-skip-rebuild.txt
+  exit 0
+fi

giteapc-config-clean.make

@@ -0,0 +1,4 @@
+# Clean the build products after install
+CONFIG_CLEAN=1
+
+export CONFIG_CLEAN

giteapc-config-noclean.make

@@ -1,4 +0,0 @@
-# Don't clean build folder after a build
-NO_CLEAN=1
-
-export NO_CLEAN

install.sh

@@ -1,33 +1,33 @@
 #! /usr/bin/env bash
 
 # Avoid rebuilds of the same version
-[[ ! -d build ]] && exit 0
+[[ -e "build/giteapc-skip-rebuild.txt" ]] && exit 0
 
 source util.sh
 PREFIX="$1"
+cd build
 
-if command -v gmake >/dev/null 2>&1; then
+if [[ -e "giteapc-build-libstdcxx.txt" ]]; then
-  make_command=gmake
+  echo "$TAG Installing libstdc++-v3 to the SuperH sysroot..."
+  run_quietly giteapc-install-libstdcxx.log \
+  $MAKE_COMMAND -j"$cores" install-strip-target-libstdc++-v3
 else
-  make_command=make
+  echo "$TAG Installing GCC to the SuperH sysroot..."
+  run_quietly giteapc-install.log \
+  $MAKE_COMMAND install-strip-gcc install-strip-target-libgcc
+
+  # Symbolic link executables to $PREFIX/bin
+  echo "$TAG Symlinking sysroot binaries to $PREFIX/bin..."
+  mkdir -p "$PREFIX/bin"
+  for f in "$SYSROOT/bin"/*; do
+    ln -sf "$f" "$PREFIX/${f#$SYSROOT/}"
+  done
 fi
 
-cd build
-echo "$TAG Installing to local folder..."
-run_quietly giteapc-install.log \
-$make_command install-strip-gcc install-strip-target-libgcc
 cd ..
 
-# Symbolic link executables to $PREFIX/bin
+# Cleanup build files after installing libstdc++
-echo "$TAG Symlinking binaries..."
+if [[ ! -z "$CONFIG_CLEAN" && -e "build/giteapc-build-libstdcxx.txt" ]]; then
-mkdir -p "$PREFIX/bin"
-for x in bin/*; do
-  ln -sf "$(pwd)/$x" "$PREFIX/$x"
-done
-
-# Cleanup build files
-if [[ -z $NO_CLEAN ]]; then
   echo "$TAG Cleaning up build files..."
-  rm -rf gcc-*/ gcc-*.tar.*
+  rm -rf gcc-*/ gcc-*.tar.* build/
-  rm -rf build/
 fi

patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch

@@ -0,0 +1,65 @@
+# Building libstdc++-v3 for a bare-bones compiler is somewhat in a gray area
+# since we don't have a full hosted system to target. From a version of GCC to
+# the next this might break in minor ways.
+#
+# Here, the issue is the reliance on a dlopen() check which cannot be
+# completed because we don't have link tests. The intent of the configure
+# script is to skip it for cross-compilers, which is materialized by the check
+# for --with-newlib (which is somewhat abused to mean "no libc").
+#
+# However, building with --with-newlib is not quite right since we have our
+# own libc and fails due to it enabling small newlib-specific bits of code
+# that don't compile with the fxlibc.
+#
+# Instead, we disable the test when --without-headers is specified, which is
+# the case in our configuration. This extra condition is already part of the
+# upstreamed source in GCC 12.1.0, so this is basically a backport. We also
+# backport disabling the GLIBCXX_CROSSCONFIG which would block the build.
+#
+# Explanations about the gray area:
+#   <https://gcc.gnu.org/legacy-ml/gcc/2008-03/msg00515.html>
+# Commit introducing this test in GCC 12.1.0:
+#   <https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=68c2e9e9234cb301e9e81792cad233a41e797792>
+# GCC bug motivating the commit:
+#   <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103866>
+
+--- gcc-11.1.0/libstdc++-v3/configure.ac	2021-04-27 12:00:16.000000000 +0200
++++ gcc-11.1.0/libstdc++-v3/configure2.ac 2022-08-15 22:53:21.730140593 +0200
+@@ -90,7 +90,7 @@
+ GLIBCXX_CONFIGURE
+ 
+ # Libtool setup.
+-if test "x${with_newlib}" != "xyes" && test "x${with_avrlibc}" != "xyes"; then
++if test "x${with_newlib}" != "xyes" && test "x${with_avrlibc}" != "xyes" && test "x${with_headers}" != "xno"; then
+   AC_LIBTOOL_DLOPEN
+ fi
+ AM_PROG_LIBTOOL
+@@ -353,7 +353,7 @@
+
+     AC_DEFINE(HAVE_ICONV)
+     AC_DEFINE(HAVE_MEMALIGN)
+-  else
++  elif test "x${with_headers}" != "xno"; then
+     GLIBCXX_CROSSCONFIG
+   fi
+
+--- gcc-11.1.0/libstdc++-v3/configure   2022-08-15 22:37:29.043460208 +0200
++++ gcc-11.1.0/libstdc++-v3/configure2  2022-08-15 22:37:03.403459840 +0200
+@@ -5903,7 +5903,7 @@
+
+
+ # Libtool setup.
+-if test "x${with_newlib}" != "xyes" && test "x${with_avrlibc}" != "xyes"; then
++if test "x${with_newlib}" != "xyes" && test "x${with_avrlibc}" != "xyes" && test "x${with_headers}" != "xno"; then
+   enable_dlopen=yes
+
+
+@@ -29266,7 +29266,7 @@
+
+     $as_echo "#define HAVE_MEMALIGN 1" >>confdefs.h
+
+-  else
++  elif test "x${with_headers}" != "xno"; then
+
+ # Base decisions on target environment.
+ case "${host}" in

uninstall.sh

@@ -3,12 +3,16 @@
 source util.sh
 PREFIX="$1"
 
-# Remove symlinks
+echo "$TAG Removing symlinks to $PREFIX/bin..."
-echo "$TAG Removing symlinks to binaries..."
+TOOLS="c++ cpp g++ gcc gcc-ar gcc-nm gcc-ranlib gcov gcov-dump gcov-tool \
-for x in bin/*; do
+  lto-dump"
-  rm "$PREFIX/$x"
+
+for t in $TOOLS; do
+  [[ -L "$PREFIX/bin/sh-elf-$t" ]] && rm "$PREFIX/bin/sh-elf-$t"
+done
+for l in "$PREFIX"/bin/sh3eb-elf-gcc-*; do
+  [[ -L "$l" ]] && rm "$l"
 done
 
-# Remove local files
+echo "$TAG Other files are managed by the fxSDK's SuperH sysroot"
-echo "$TAG Removing installed files..."
+echo "$TAG Uninstall Lephenixnoir/fxsdk to clean up the sysroot"
-rm -rf bin/ include/ lib/ libexec/ share/

util.sh

@@ -1,5 +1,21 @@
 TAG="<sh-elf-gcc>"
 
+if command -v gmake >/dev/null 2>&1; then
+  MAKE_COMMAND=gmake
+else
+  MAKE_COMMAND=make
+fi
+
+if ! command -v fxsdk >/dev/null 2>&1; then
+  echo "$TAG error: fxSDK is not installed"
+  exit 1
+elif ! fxsdk path sysroot >/dev/null 2>&1; then
+  echo "$TAG error: need fxSDK ≥ 2.9 with 'path' command"
+  exit 1
+fi
+
+SYSROOT="$(fxsdk path sysroot)"
+
 run_quietly() {
   out="$1"
   shift 1