Home > Boards > OPNsense on NanoPI 4SE

OPNsense on NanoPI 4SE

February 7th, 2023
Goto comments Leave a comment

Guide to install OPNsense ARM-edition on small router.

This guide cover an attempt to power the small computer with OPNsense system image. The target placement for the system is the internal eMMC module. This selection is made with keeping in mind the most important thing: no one microSD card can not present a significant I/O speed due to nature of operation system with base requirement for a lot of small input/output transactions. In other words, any OS wants to read and especially write a tons of small files and data. May be not exactly the “OS” (kernel), but the subsystems of OS. Log files, configurations, working artefacts on so on. This is more important for OPNsense system which is main purpose – work with networks, routing networks, filtering networks and of course – collecting a telemetry for what is going on while these processes is on going.

Table of contents

  1. Prerequisites
  2. Understanding u-boot “partitioning”
  3. Preparing bootable image
  4. Boot the R4SE device with freshly prepared image
  5. Install on eMMC
  6. Patching eMMC partitions footprint for u-boot requirements

Prerequisites

  • microSD card enough capacity to fit an OPNSense image (8 gigabytes is fine enough)
  • (optional) TTL/UART adapter to establish serial console connection
  • Linux or BSD powered computer
  • time 🙂

Understanding u-boot “partitioning”

To be able for booting with using u-boot loader, first we need to understand for “u-boot” is a complex boot-loader which is consists of two parts, placed in different places of bootable drive’s surface. Describing for roots of purpose for each of u-boot’s parts is out of the scope for current letter. There we need to consolidate the next understanding: each part of u-boot loader need to be placed into the specific area of drive and none of the “neighbors” should newer overwrite them.
This means, that the area for u-boot placement should be formed as a separate “slice” of the disk surface. Easy way to reach this – is a reserve for dedicated free space before any other partitions placed into the drive surface.
This is the document where mentioned above is claimed as a technical table:
https://opensource.rock-chips.com/wiki_Partitions
“Two parts” of u-boot loader in a terms of this document is a “loader1” and “loader2” partitions. Thus, we need to guarantee to reserve a space for first 16 megabytes of the disk surface and place any partitions only after that gap.

Preparing bootable image

The author of OPNsense images are not prepare dedicated image for R4SE device. This is not a problem – we can made dedicated image by ourself. “Dedicated” – means that image contain an adapted u-boot loaders especially for desired hardware. This is by nature of u-boot: every sample of u-boot is “designed” only for the hardware on which it should run loading an OS. In other words, trying to boot a device with wrong u-boot will lead a “bricked” device after boot process is done or device run into the boot-loop.

Obtain requirements
  1. Download an image.
    wget https://personalbsd.org/images/OPNsense-22.1.9-OpenSSL-aarch64-NanoPi-R4S-20220626.img.xz

  2. Unpack the image.
    unxz -v OPNsense-22.1.9-OpenSSL-aarch64-NanoPi-R4S-20220626.img.xz

    Resulting image file after unpack are done is
    OPNsense-22.1.9-OpenSSL-aarch64-NanoPi-R4S-20220626.img

  3. Download dedicated for R4SE u-boot package.
    wget http://pkg.personalbsd.org/FreeBSD:13:aarch64-default/All/u-boot-nanopi-r4se-2020.07.pkg

  4. Unpack the archive.
    tar xvzf u-boot-nanopi-r4se-2020.07.pkg

    After unpack the u-boot bodies are placed into the path: usr/local/share/u-boot/u-boot-nanopi-r4se/
    Be aware, that the path is the relative path from the your current working directory
    Inside this directory you will find next two files:
    idbloader.img
    u-boot.itb

    Where idbloader.img is the “loader1” and u-boot.itb is the “loader2” in the terms of u-boot “partitions”

Work with image (BSD)

If you have BSD equipped computer, this section is for use. If not – skip to next section describing universal steps

  1. Mount an image as root
    sudo mdconfig OPNsense-22.1.9-OpenSSL-aarch64-NanoPi-R4S-20220626.img

    this command after done should print the name of fresh mounted “device”. For example:
    md0

  2. Rewrite the u-boot parts
    sudo dd if=usr/local/share/u-boot/u-boot-nanopi-r4se/idbloader.img of=/dev/md0 seek=64 bs=512 conv=sync

    After a short while we got a result:
    353+0 records in
    353+0 records out
    180736 bytes transferred in 0.436434 secs (414120 bytes/sec)

    sudo dd if=usr/local/share/u-boot/u-boot-nanopi-r4se/u-boot.itb of=/dev/md0 seek=16384 bs=512 conv=sync

    After a short while we got a result:
    1688+0 records in
    1688+0 records out
    864256 bytes transferred in 1.960308 secs (440878 bytes/sec)

  3. Finish works with an image:
    sudo mdconfig -d -u 0

  4. Burn an image onto the SD-card
    sudo dd if=./OPNsense-22.1.9-OpenSSL-aarch64-NanoPi-R4S-20220626.img of=/dev/da0 bs=4096 status=progress conv=sync

    After a while we got a result:
    786432+0 records in
    786432+0 records out
    3221225472 bytes transferred in 3386.892073 secs (951086 bytes/sec)

Work with image (universal)

This is universal steps for any Linux/BSD computer

  1. Burn an image onto the SD-card (assume we have SD card appeared in the system as da0 device)
    sudo dd if=./OPNsense-22.1.9-OpenSSL-aarch64-NanoPi-R4S-20220626.img of=/dev/da0 bs=4096 status=progress conv=sync

    After a while we got a result:
    786432+0 records in
    786432+0 records out
    3221225472 bytes transferred in 3386.892073 secs (951086 bytes/sec)

  2. Rewrite the u-boot parts
    sudo dd if=usr/local/share/u-boot/u-boot-nanopi-r4se/idbloader.img of=/dev/da0 seek=64 bs=512 conv=sync

    After a short while we got a result:
    353+0 records in
    353+0 records out
    180736 bytes transferred in 0.436434 secs (414120 bytes/sec)

    sudo dd if=usr/local/share/u-boot/u-boot-nanopi-r4se/u-boot.itb of=/dev/da0 seek=16384 bs=512 conv=sync

    After a short while we got a result:
    1688+0 records in
    1688+0 records out
    864256 bytes transferred in 1.960308 secs (440878 bytes/sec)

Boot the R4SE device with freshly prepared image

  1. Put SD-card with burned image into the Nano pi R4SE
  2. Plug Ethernet-cabled
  3. (optional) Plug the Serial adapter
  4. Hold the MASK button with the pen
  5. Power the R4SE device by the USB-C cable
  6. Release the MASK button
  7. If we have pluged in a serial adapter, watch the boot process from SD-card (examples attached below)
  8. After boot is done, enter into the WEB-Gui and perform an initial configuration

u-boot loader output:

sd-ubootDownload

Efi-loader output:

efiDownload

BSD-loader output:

loaderDownload

BSD-kernel output:

kernelDownload

Install on eMMC

  1. Inside WEB-Gui, enable ssh. How to perform that is showed on followed image:

Required to modify options is marked with the red rectangle.

  1. Establish connection with the R4SE device within the ssh
    ssh root@192.168.1.1

    when establishing an initial connection, we can face with the following:
    The authenticity of host '192.168.1.1 (192.168.1.1)' can't be established.
    ED25519 key fingerprint is SHA256:[long-hexadecimal-string-with-key-fingerprint].
    No matching host key fingerprint found in DNS.
    This key is not known by any other names
    Are you sure you want to continue connecting (yes/no/[fingerprint])?

    We need to answer “yes” by manually typing a letters and press “Enter”
    Right after the answer, system inform us about adding the key in well-known-hosts file and prompts us to enter for the root password:
    Warning: Permanently added '192.168.1.1' (ED25519) to the list of known hosts.
    (root@192.168.1.1) Password:

    initial password’s value is “opnsense”, so after entering the password next menu are shown:

  1. Reach the shell by selecting item 8 in the menu showed above
  2. Prepare the eMMC device with the command:
    gpart destroy -F mmcsd1

    Command execution will give the output:
    mmcsd1 destroyed

  3. Launch an OPNsense installation process by exec the command:
    opnsense-installer

  4. Now, we need to proceed within the installation process shown on the following steps

This is the initial step. We can setup the desired keymap.
There we need to select a custom install due to some adjusting options. SWAP size for example
Proceed with ZFS/Root-on-ZFS mode
Update the SWAP size and proceed to select pool type and disk selection
Type of ZFS configuration is the “stripe”
Select an our eMMC module here
Now we ready to perform an installation itself
Last chance to …
There is a progress of installation job
Done. After selection “Exit” be ready to press Ctrl-C combination. There only frame with five seconds length to press.
  1. Once installation process is finished, right after exit from the installer program, we need to press Ctrl-C keys combination and exit to the shell

Patching eMMC partitions footprint for u-boot requirements

Due to installer of OPNsense system is know nothing about u-boot, we need to update partitions placement.

  1. Perform EFI partition backup with the following set of commands:
    mkdir /mnt/efi
    mount_msdosfs /dev/mmcsd1p1 /mnt/efi
    cp -Rv /mnt/efi /root
    /mnt/efi -> /root/efi
    /mnt/efi/efi -> /root/efi/efi
    /mnt/efi/efi/freebsd -> /root/efi/efi/freebsd
    /mnt/efi/efi/freebsd/loader.efi -> /root/efi/efi/freebsd/loader.efi
    /mnt/efi/efi/boot -> /root/efi/efi/boot
    /mnt/efi/efi/boot/bootaa64.efi -> /root/efi/efi/boot/bootaa64.efi

    umount /mnt/efi

  2. Destroy the EFI partition:
    gpart delete -i 1 mmcsd1

    When done system prints out:
    mmcsd1p1 deleted

  3. Create back EFI partition but with the shrinked size and adjusted starting place
    gpart add -t efi -b 16M -s 244M -i 1 -l EFI mmcsd1

    When done system prints out:
    mmcsd1p1 added

    newfs_msdos -F 16 -L EFI /dev/mmcsd1p1

  4. Restore the EFI partition from previously created backup
    mount_msdosfs /dev/mmcsd1p1 /mnt/efi
    mv -v /root/efi/* /mnt/efi/

    /root/efi/efi -> /mnt/efi/efi
    /root/efi/efi/freebsd -> /mnt/efi/efi/freebsd
    /root/efi/efi/freebsd/loader.efi -> /mnt/efi/efi/freebsd/loader.efi
    /root/efi/efi/boot -> /mnt/efi/efi/boot
    /root/efi/efi/boot/bootaa64.efi -> /mnt/efi/efi/boot/bootaa64.efi

    umount /mnt/efi

  5. Enable debug flag on filesystem
    sysctl kern.geom.debugflags=16

  6. Copy the u-boot from booted SD-card into the eMMC placement
    dd if=/dev/mmcsd0 of=/dev/mmcsd1 bs=512 skip=64 seek=64 count=7104 conv=sync

    After a short while we got a result:
    7104+0 records in
    7104+0 records out
    3637248 bytes transferred in 18.275839 secs (199019 bytes/sec)

    dd if=/dev/mmcsd0 of=/dev/mmcsd1 bs=512 skip=16384 seek=16384 count=8192 conv=sync

    After a short while we got a result:
    8192+0 records in
    8192+0 records out
    4194304 bytes transferred in 7.278221 secs (576281 bytes/sec)

  7. Shut down the R4SE device
    shutdown -h now

    Wait until the device is stop responding (for example on a ping 192.268.1.1 command)

  8. Unplug the power cable
  9. Remove the SD-card from R4SE device
  10. Plug the power cable back
  11. Boot process will start from the eMMC module. If you have the serial adapter cable plugged in, you can see the difference in outputs which is produced by the u-boot loader. The difference is the device used to boot-process.

U-boot output when boot starts from eMMC module:

emmc-ubootDownload

The end.

Prepared by A.Petrov

Boards

  1. No comments yet.
  1. No trackbacks yet.