[FOTA] Local-OTA for Embedded Linux System
最后更新于
最后更新于
The Local-OTA, as the name implies, is a scheme for upgrading the platform locally. This scheme is the basis of FOTA and can be integrated into FOTA.
OSTree provides a number of very significant technological advantages over other full-filesystem updating schemes. For embedded systems that need a solution for safe, atomic, full-filesystem updates, the usual approach is to have some kind of dual-bank scheme. Here, we’re going to take a look at the difference between OSTree and dual-bank systems, and the advantages OSTree can provide.
The work progress is shown in the following table:
Steps | Content | Status |
1 | Basic Knowledge | Done |
2 | Create remote repository | Done |
3 | Setup HTTP server for remote repository | Done |
4 | Transplant OSTree tool to embedded device | Done |
5 | Deloy the ostree ROOTFS metadata on embedded device | Done |
6 | Enabling the ostree boot metadata on embedded device | Done |
7 | Developement the RAMDISK for embedded device | Done |
In general, there are 2 methods for performing updates to an end node follows:
Method 1: Dual-bank system:
Note, The firmware contains the Linux Kernel, ramdisk, Linux Device Tree, and Linux RootFS.
Method 2: Update is performed on top of the existing version
In a dual-bank system, the read-only root filesystem is kept on a different partition from the writable user space, so that when an update is needed the whole partition can be overwritten. For atomicity and safety, this read-only partition is duplicated: there are two complete copies of the filesystem, kept on different partitions, and the active partition can be selected at boot time.
When the system needs to be updated, the new filesystem image is written to the inactive partition, and the next time the system reboots, that partition becomes the active one.
The main advantage of this updated model is its safety. Updates are always strictly atomic, and there is always a known good image that can be rolled back to. However, there are significant trade-offs in flexibility and materials costs that must be made: the size of the root partition must be chosen when the system is flashed for the very first time, and the duplication of the root partition doubles the space required. When choosing how big to make the root partition, a device manufacturer has to consider not just how big their filesystem image currently is, but also must estimate and plan for the size of all future updates. If the size chosen is too small, it may restrict the ability to add new features. Making it larger, of course, adds to the bill of goods for the product—and since it’s duplicated, every extra megabyte of future capacity actually costs two megabytes to accommodate.
OSTree checksums individual files and stores them as content-addressed objects, much like git. The read-only filesystem is built by "checking out" a particular revision and hard-linking the content-addressed objects into the actual Linux directory structure. Multiple filesystem versions can be stored, and any content that is duplicated across versions is only stored once. A complete history of all versions is stored in TreeHub, but it is not required to store that complete revision history on the device. Only one partition is needed—writable user space can be on the same partition as the OSTree content store.
When an embedded system needs to be updated, the Treehub (remote repository) sends a small metadata file with a particular commit identifier. The device pulls that commit from Treehub (remote repository), only downloading the new files, and only downloading binary diffs of changed files. Once the pull is complete and verified, the system is instructed to boot into the new version the next time it starts up.
With OSTree, you no longer need to guess how much room you might need in the future to expand your system; the OSTree content store expands and contracts as needed. You also save a significant amount of space, since only diffs between versions need to be stored. OSTree also allows you to garbage-collect old images: if you upgrade 1.0 → 1.1 → 1.2, for example, by default the OTA Connect client will garbage-collect all local objects unique to 1.0. If you decided later on that you in fact did want to go back to v1.0, you still could: if you pushed v1.0 from the remote repository, the device would download only the diff from the remote repository, repopulate the local object store, and then reboot into that version. Of course, it’s also possible to configure OSTree to keep more than two revisions on the local disk.
A demo of the NXP-IMX6ULL platform with HTTP downloading the firmware is shown as the link: https://youtu.be/3i48NbAS2jU?t=1659
The updating process is shown in the following figure:
OSTree checksums individual files and stores them as content-addressed objects, much like git. The OSTree Tool and OSTree library are in the link https://github.com/ostreedev/ostree.
The meta-updater is in the Linux ramdisk and it is an open-source tool, please refer to the link: https://github.com/advancedtelematic/meta-updater
The following steps illustrate the deploy the OSTree for embedded devices:
Setup Remote Repository.
Integrate the Repository.
Embedded Device Deployment.
Remote repository usage.
To create an OSTree repository, you can follow these steps:
Install the ostree package on your system if it is not already installed. This package contains the necessary tools to manage an OSTree repository.
Create an empty directory to serve as the root of your repository. This directory will contain all the objects and metadata needed for the repository.
mkdir my-ostree-repo
Initialize the repository using the ostree init command. This will create the necessary metadata files and directories.
ostree init --mode=archive-z2 --repo=my-ostree-repo
The --mode=archive-z2 option specifies the compression format to use for the repository. You can choose a different format if you prefer.
Add content to the repository using the ostree commit command. This command takes a directory or a file as input and creates a new commit in the repository.
ostree commit --branch=my-branch --repo=my-ostree-repo /path/to/content
The --branch option specifies the name of the branch to create, and the /path/to/content parameter specifies the directory or file to include in the commit.
Repeat step 4 for each piece of content you want to add to the repository.
Publish the repository using the ostree summary and ostree summary-to-filename commands. These commands create a summary file that describes the contents of the repository and allows clients to quickly check for updates.
$ ostree summary --repo=my-ostree-repo
$ ostree summary-to-filename --repo=my-ostree-repo > my-ostree-repo/refs/heads/my-branch.summary
Make the repository available to clients by serving it over HTTP or HTTPS. You can use any web server to serve the repository directory.
sudo python3 -m http.server 80 --bind 0.0.0.0 --directory my-ostree-repo/
This command serves the repository directory on port 80 and allows any client to access it. You should use a different port and restrict access to trusted clients in a production environment.
In the main storage partition, we have basically two directories that are mounted at /sysroot:
boot directory (/boot)
OSTree repository (/ostree)
The embedded device filesystem trees (also called deployments) are checked out at /sysroot/ostree/deploy/<os>/deploy/<commit> (files there are just hard links to objects in the repository)
A deployment is bind-mounted as a read-write rootfs at /, and the /usr directory from the deployment is bind-mounted read-only at /usr. A deployment is shown in the following figure:
The storage device is the deployed repository.
The runtime is the embedded device filesystem.
To integrate an OSTree repository, you can follow these steps:
Generate the sysroot partition with the boot directory (/boot) and the OSTree repository (/ostree).
Prepare the default deployment in /sysroot/ostree/deploy/deploy/.
Make sure U-Boot will be able to load and boot the kernel artifacts (kernel image, device tree, ramdisk).
Boot a ramdisk image that will mount the OSTree deployment and switch to it. The step is already (mostly) implemented in meta-updater, please refer to https://docs.ota.here.com/ota-client/latest/bsp-integration.html
A device’s SD card or eMMC partition is shown in the following figure:
Boot Directory: The boot partition stores the data of the last three boot images.
OSTree Repository: The ostree stores the OSTree repository metadata.
System Root: the sysroot is the real root partition used. The ramdisk will chroot() to this path.
In the ramdisk, we can backup the sysroot to sysroot.old, and checkout the newest rootfs from the repository by ostree checkout --repo=./ostree/repo master sysroot. If there is no issue in the newest rootfs, the sysroot.old can be dropped from the filesystem. Finally, the ramdisk boots the sysroot as the rootfs.
The ostree remote command is used to manage OSTree repositories. OSTree is a tool for managing bootable, immutable file systems, and the remote command specifically deals with adding, removing, and listing remote repositories.
Here are some examples of how to use the ostree remote command:
To add a remote repository:
ostree remote add --set=gpg-verify=false --no-gpg-verify myrepo https://example.com/repo
This command adds a remote repository named myrepo located at https://example.com/repo and sets gpg-verify to false.
To remove a remote repository:ostree remote delete myrepo
This command removes the remote repository named myrepo.
To list remote repositories:ostree remote list
This command lists all the remote repositories configured on the system.
To show details of a remote repository:ostree remote show myrepo
This command shows the details of the remote repository named myrepo, such as its URL, GPG verification settings, and any proxies or authentication configured.
It is difficult to cross-compile the OSTree in the ARM platform (ARMv7/v8) because the ostree is based on various libraries. But there are multiple friendly build systems out there, such as Buildroot and Yocto for example. There are also ostree recipes for the latter at least. I'd recommend looking into one of those instead of a hand-rolling one.
The buildroot (https://buildroot.org/) is a simple, efficient, and easy-to-use tool to generate embedded Linux systems through cross-compilation. We can utilize the buildroot to help us to generate the ostree binary and libraries on the embedded Linux system.
For the ZYNQ platform, you can refer to the link https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842369/Build+Linux+for+Zynq-7000+AP+SoC+using+Buildroot.
For the S32g platform, you can refer to the link S32G Linux Boot Time Optimization
For the Layerscape platform, you can refer to the Layerscape Software Development Kit User Guide.
You should configure the platform by the corresponding BSP user guide above as step 1. The next step should add the ostree and its net libraries using the menuconfig.
In order to the HTTP function, the libsoup and libcurl libraries shall be also enabled.
The libcurl set by:
The libsoup set by:
The ostree can be run on the embedded device.
In this demo, we make use of the output of the buildroot directly. The ~/temp mocks the remote root repository, and we create a repo directory as the remote ostree repository.
mkdir -p repo && ostree --repo=repo init --mode=archive-z2 && mkdir -p rootfs
Add an empty rootfs to the repository and commit it as the V1 version.
ostree --repo=repo commit --branch=master --subject="image v1 (empty)" rootfs/
Copy the rootfs of buildroot output to the rootfs path, the rootfs layout is shown in the following figure:
We commit the new rootfs as the “image v2“
ostree --repo=repo commit --branch=master --subject="image v2 (zynq rootfs)" rootfs/
We can list the commit records by ostree log master --repo=repo
Publish the repository using the ostree summary and ostree summary-to-filename commands. These commands create a summary file that describes the contents of the repository and allows clients to quickly check for updates.
ostree summary --repo=repo ./repo/branch.summary -u
In this demo, we use the python http server to mock the remote server by:
python3 -m http.server 8000 --bind 192.168.32.2 --directory repo
The SD rootfs (/dev/mmcblk0p2) file structure is shown in the following figure:
We should init the repository for the ostree repo in the SD card.
ostree --repo=repo init --mode=archive-z2
Add a remote address for the ostree repo
ostree remote add --no-gpg-verify origin http://192.168.32.2:8000/
Pull the metadata for ostree repo
ostree remote pull origin:master
Check out the newest rootfs by
ostree checkout --repo=repo master sysroot
The meta-updater is in the ramdisk, its function:
Detect if an upgrade is required
Update the boot.1 files to boot partition (/dev/mmcblk0p1)
Pull and Checkout the newest rootfs from the ostree repository.
Boot the new rootfs
In the ramdisk init script, we add
The new rootfs can recover from the repository.
