# \[FOTA] Local-OTA for Embedded Linux System The Local-OTA, as the name implies, is a scheme for upgrading the platform locally. This scheme is the basis of FOTA and can be integrated into FOTA. **OSTree** provides a number of very significant technological advantages over other full-filesystem updating schemes. For embedded systems that need a solution for safe, atomic, full-filesystem updates, the usual approach is to have some kind of **dual-bank** scheme. Here, we’re going to take a look at the difference between OSTree and dual-bank systems, and the advantages OSTree can provide. The work progress is shown in the following table: | **Steps** | **Content** | **Status** | | --------- | ---------------------------------------------------- | ---------- | | 1 | Basic Knowledge | Done | | 2 | Create remote repository | Done | | 3 | Setup HTTP server for remote repository | Done | | 4 | Transplant OSTree tool to embedded device | Done | | 5 | Deloy the ostree ROOTFS metadata on embedded device | Done | | 6 | Enabling the ostree boot metadata on embedded device | Done | | 7 | Developement the RAMDISK for embedded device | Done | ## 1. Comparing full-filesystem update strategies In general, there are 2 methods for performing updates to an end node follows: **Method 1**: Dual-bank system:

Note, The firmware contains the Linux Kernel, ramdisk, Linux Device Tree, and Linux RootFS. **Method 2**: Update is performed on top of the existing version

### 1.1 Dual-bank In a dual-bank system, the read-only root filesystem is kept on a different partition from the writable user space, so that when an update is needed the whole partition can be overwritten. For atomicity and safety, this read-only partition is duplicated: there are two complete copies of the filesystem, kept on different partitions, and the active partition can be selected at boot time. When the system needs to be updated, the new filesystem image is written to the inactive partition, and the next time the system reboots, that partition becomes the active one.

The main advantage of this updated model is its **safety**. Updates are always strictly **atomic**, and there is always a known good image that can be **rolled back to**. However, there are significant trade-offs in flexibility and **materials costs** that must be made: the size of the root partition must be chosen when the system is flashed for the very first time, and the **duplication of the root partition doubles the space required**. When choosing how big to make the root partition, a device manufacturer has to consider not just how big their filesystem image currently is, but also must estimate and plan for the size of all future updates. If the size chosen is too small, it may restrict the ability to add new features. Making it larger, of course, adds to the bill of goods for the product—and since it’s duplicated, every extra megabyte of future capacity actually costs two megabytes to accommodate. ### 1.2 Single on top of the existing version (OSTree) OSTree checksums individual files and stores them as content-addressed objects, much like git. The read-only filesystem is built by "checking out" a particular revision and hard-linking the content-addressed objects into the actual Linux directory structure. **Multiple filesystem versions can be stored**, and any content that is duplicated across versions is only **stored once**. A complete history of all versions is stored in **TreeHub**, but it is not required to store that complete revision history on the device. Only one partition is needed—writable user space can be on the same partition as the OSTree content store.

When an embedded system needs to be updated, the Treehub (remote repository) sends a **small metadata file** with a particular commit identifier. The device pulls that commit from Treehub (remote repository), only downloading the new files, and only downloading binary diffs of changed files. Once the pull is complete and verified, the system is instructed to boot into the new version the next time it starts up. With OSTree, you no longer need to guess how much room you might need in the future to expand your system; the OSTree content store expands and contracts as needed. You also save a significant amount of space, since only diffs between versions need to be stored. OSTree also allows you to garbage-collect old images: if you upgrade 1.0 → 1.1 → 1.2, for example, by default the OTA Connect client will garbage-collect all local objects unique to 1.0. If you decided later on that you in fact did want to go back to v1.0, you still could: if you pushed v1.0 from the remote repository, the device would download only the diff from the remote repository, repopulate the local object store, and then reboot into that version. Of course, it’s also possible to configure OSTree to keep more than two revisions on the local disk. A demo of the NXP-IMX6ULL platform with HTTP downloading the firmware is shown as the link: ## 2. OSTree Method The updating process is shown in the following figure:

OSTree checksums individual files and stores them as content-addressed objects, much like git. The OSTree Tool and OSTree library are in the link . The meta-updater is in the Linux ramdisk and it is an open-source tool, please refer to the link: ### 2.1 OSTree Steps The following steps illustrate the deploy the OSTree for embedded devices: * Setup Remote Repository. * Integrate the Repository. * Embedded Device Deployment. * Remote repository usage. #### 2.1.1 Setup Remote Repository To create an OSTree repository, you can follow these steps: * Install the `ostree` package on your system if it is not already installed. This package contains the necessary tools to manage an OSTree repository. * Create an empty directory to serve as the root of your repository. This directory will contain all the objects and metadata needed for the repository. * `mkdir my-ostree-repo` * Initialize the repository using the `ostree init` command. This will create the necessary metadata files and directories. * `ostree init --mode=archive-z2 --repo=my-ostree-repo` The `--mode=archive-z2` option specifies the compression format to use for the repository. You can choose a different format if you prefer. * Add content to the repository using the `ostree commit` command. This command takes a directory or a file as input and creates a new commit in the repository. * `ostree commit --branch=my-branch --repo=my-ostree-repo /path/to/content` The `--branch` option specifies the name of the branch to create, and the `/path/to/content` parameter specifies the directory or file to include in the commit. * Repeat step 4 for each piece of content you want to add to the repository. * Publish the repository using the `ostree summary` and `ostree summary-to-filename` commands. These commands create a summary file that describes the contents of the repository and allows clients to quickly check for updates. \ $ `ostree summary --repo=my-ostree-repo`\ `$ ostree summary-to-filename --repo=my-ostree-repo > my-ostree-repo/refs/heads/my-branch.summary` * Make the repository available to clients by serving it over HTTP or HTTPS. You can use any web server to serve the repository directory. * `sudo python3 -m http.server 80 --bind 0.0.0.0 --directory my-ostree-repo/`\ This command serves the repository directory on port 80 and allows any client to access it. You should use a different port and restrict access to trusted clients in a production environment. #### 2.1.2 Repository Integration In the main storage partition, we have basically two directories that are mounted at `/sysroot`: * boot directory (`/boot`) * OSTree repository (`/ostree`) The embedded device filesystem trees (also called deployments) are checked out at `/sysroot/ostree/deploy//deploy/` (files there are just hard links to objects in the repository) A deployment is bind-mounted as a read-write rootfs at `/`, and the `/usr` directory from the deployment is bind-mounted read-only at `/usr`. A deployment is shown in the following figure:

* The storage device is the deployed repository. * The runtime is the embedded device filesystem. To integrate an OSTree repository, you can follow these steps: * Generate the `sysroot` partition with the boot directory (`/boot`) and the OSTree repository (`/ostree`). * Prepare the default deployment in `/sysroot/ostree/deploy/deploy/`. * Make sure U-Boot will be able to load and boot the kernel artifacts (kernel image, device tree, ramdisk). * Boot a ramdisk image that will mount the OSTree deployment and switch to it. The step is already (mostly) implemented in meta-updater, please refer to #### 2.1.3 Deployment A device’s SD card or eMMC partition is shown in the following figure:

* Boot Directory: The `boot` partition stores the data of the last three boot images. * OSTree Repository: The `ostree` stores the OSTree repository metadata. * System Root: the `sysroot` is the real root partition used. The ramdisk will `chroot()` to this path. In the ramdisk, we can backup the `sysroot` to `sysroot.old`, and *checkout* the newest rootfs from the repository by `ostree checkout --repo=./ostree/repo master sysroot`. If there is no issue in the newest rootfs, the `sysroot.old` can be dropped from the filesystem. Finally, the ramdisk boots the `sysroot` as the rootfs. #### 2.1.4 Usage Remote Repository The `ostree remote` command is used to manage OSTree repositories. OSTree is a tool for managing bootable, immutable file systems, and the `remote` command specifically deals with adding, removing, and listing remote repositories. Here are some examples of how to use the `ostree remote` command: **To add a remote repository:** `ostree remote add --set=gpg-verify=false --no-gpg-verify myrepo https://example.com/repo` This command adds a remote repository named `myrepo` located at `https://example.com/repo` and sets `gpg-verify` to false. **To remove a remote repository**:`ostree remote delete myrepo` This command removes the remote repository named `myrepo`. **To list remote repositories:**`ostree remote list` This command lists all the remote repositories configured on the system. **To show details of a remote repository**:`ostree remote show myrepo` This command shows the details of the remote repository named `myrepo`, such as its URL, GPG verification settings, and any proxies or authentication configured. ``` ostree remote add --no-gpg-verify origin http://127.0.0.1:8000/ ostree remote show-url origin --repo=repo ostree remote refs origin ostree remote pull origin:master ostree log origin:master ``` ### 3. OSTree Transplanting It is difficult to cross-compile the OSTree in the ARM platform (ARMv7/v8) because the ostree is based on various libraries. But there are multiple friendly build systems out there, such as Buildroot and Yocto for example. There are also ostree recipes for the latter at least. I'd recommend looking into one of those instead of a hand-rolling one. The buildroot () is a simple, efficient, and easy-to-use tool to generate embedded Linux systems through cross-compilation. We can utilize the buildroot to help us to generate the ostree binary and libraries on the embedded Linux system. * For the ZYNQ platform, you can refer to the link . * For the S32g platform, you can refer to the link[ S32G Linux Boot Time Optimization](https://community.nxp.com/pwmxy87654/attachments/pwmxy87654/nxp-designs%40tkb/682/2/S32G_Optimize_Linux_BootTime_V2-20220124_Eng.pdf) * For the Layerscape platform, you can refer to the [Layerscape Software Development Kit User Guide.](https://www.nxp.com/docs/en/user-guide/LSDKUG_Rev19.09.pdf) ### 3.1 Config You should configure the platform by the corresponding BSP user guide above as step 1. The next step should add the ostree and its net libraries using the `menuconfig`. #### 3.1.1 OSTree package

#### 3.1.2 libsoup and libcurl In order to the HTTP function, the libsoup and libcurl libraries shall be also enabled.

The libcurl set by:

The libsoup set by:

The ostree can be run on the embedded device.

## 4. Demo ### 4.1 Preparing remote rootfs repository In this demo, we make use of the output of the buildroot directly. The `~/temp` mocks the remote root repository, and we create a `repo` directory as the remote ostree repository. `mkdir -p repo && ostree --repo=repo init --mode=archive-z2 && mkdir -p rootfs` Add an empty rootfs to the repository and commit it as the `V1` version. `ostree --repo=repo commit --branch=master --subject="image v1 (empty)" rootfs/`

Copy the rootfs of buildroot output to the `rootfs` path, the rootfs layout is shown in the following figure:

We commit the new rootfs as the “image v2“ `ostree --repo=repo commit --branch=master --subject="image v2 (zynq rootfs)" rootfs/`

We can list the commit records by `ostree log master --repo=repo`

Publish the repository using the `ostree summary` and `ostree summary-to-filename` commands. These commands create a summary file that describes the contents of the repository and allows clients to quickly check for updates. `ostree summary --repo=repo ./repo/branch.summary -u` In this demo, we use the python http server to mock the remote server by: `python3 -m http.server 8000 --bind 192.168.32.2 --directory repo` ### 4.2 Creating the device rootfs The SD rootfs (/dev/mmcblk0p2) file structure is shown in the following figure:

We should init the repository for the ostree repo in the SD card. `ostree --repo=repo init --mode=archive-z2` Add a remote address for the ostree repo `ostree remote add --no-gpg-verify origin http://192.168.32.2:8000/` Pull the metadata for ostree repo `ostree remote pull origin:master`

Check out the newest rootfs by `ostree checkout --repo=repo master sysroot` ### 4.3 meta-updater The meta-updater is in the ramdisk, its function: * Detect if an upgrade is required * Update the boot.1 files to boot partition (`/dev/mmcblk0p1`) * Pull and Checkout the newest rootfs from the ostree repository. * Boot the new rootfs In the ramdisk init script, we add ``` echo "[FOTA-meta-updater] mount -o rw /dev/mmcblk0p2 /mnt" mount -o rw /dev/mmcblk0p2 /mnt if [ $? -eq 0 ]; then echo "[INFO] mount done." else echo "[ERR] mount failed." exit -1 fi echo "[FOTA-meta-updater] delete old rootfs" rm -rf /mnt/sysroot sync echo "[FOTA-meta-updater] ostree update the newest rootfs from the repo" ostree checkout master /mnt/sysroot --repo=/mnt/repo if [ $? -eq 0 ]; then echo "[INFO] ostree checkout master /mnt/sysroot done." else echo "[ERR] ostree checkout master /mnt/sysroot failed." exit -1 fi echo "[FOTA-meta-updater] ostree updated!!!!!!!!!!!" echo "[FOTA-meta-updater] remount the new rootfs" mount --bind /mnt/sysroot $ROOTFS_DIR if [ $? -eq 0 ]; then echo "[INFO] mount bind done." else echo "[ERR] mount bind failed." exit -1 fi echo "[FOTA-meta-updater] mounted new rootfs" echo "[FOTA-meta-updater] booting from the new rootfs: " echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." echo "[FOTA-meta-updater] ..........................................." ```

The new rootfs can recover from the repository. ## Ref * * * * * * * * * * * * * * * * [Preparing the Host and Platform Project for OSTree Support](https://docs.windriver.com/bundle/Wind_River_Linux_Tutorial_Using_OSTree_for_Updates_at_Run_Time_LTS_19/page/vpu1573077211787.html) --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://carloss-organization-4.gitbook.io/tech/design/fota/fota-local-ota-for-embedded-linux-system.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.