Difference between revisions of "Poky manual"

Latest revision as of 11:22, 1 June 2022

Description

Build the firmware for Elphel 10393 series camera systems from scratch.

Prebuilt releases can be found here.

Use cases

Update Linux Kernel when developing a driver
Include application to the default firmware package
Update Elphel's PHP extension
C/C++ applications if developed on PC require a cross-compiler and require Yocto installation or some other alternative.

Other projects written in Python, PHP, Javascript, HTML, C/C++ (gcc is installed on the system board) can be developed on the camera. Installing the framework is not required but it can simplify the development process.

Description

Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
Yocto Poky revision = 2.7.1 (Warrior) - builds in Kubuntu 20.04. Older OSes are not supported. Roll back to get built in 16.04.
~~Yocto Poky revision = 2.4 (Rocko) - builds in Kubuntu 16.04. Won't build in 18.04 or 20.04.~~
~~Yocto Poky revision = 2.0 (Jethro)~~

Features

Environment is set up with a single script
All Elphel projects are configured for Eclipse IDE:
- fpga projects (verilog)
- kernel
- applications projects
All steps for fpga projects are run from Eclipse IDE
- Read about setting up Eclipse based FPGA development
- VDT Plugin repository

Once everything is built it supports updating software projects (files and binaries) on the target system
- access is set up in yocto's local.conf (see setup.py output) + ssh-copy-id
- bitbake some-app -c target_scp from the command line or Eclipse IDE - see examples in elphel-apps-... at Git

Requirements

Kubuntu 20.04 x64
~~Kubuntu 18.04 x64 (as of Dec 2018) - Poky: 2.7.1 Warrior (not tested but might build w/o problems)~~
~~Kubuntu 16.04 x64 (as of Dec 2018) - Poky: 2.4 Rocko, 2.7.1 Warrior~~

Output files

The names are listed as they appear in the u-boot configuration header file - actual output files have different names:

File(s)	Build Command	Description
boot.bin	bitbake u-boot	u-boot as the first stage bootloader = Secondary Program Loader (SPL) that boots u-boot-dtb.img
u-boot-dtb.img	bitbake u-boot	full size u-boot with a stripped device tree (cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img)
devicetree.dtb	bitbake device-tree	device tree with listed interfaces, zynq registers, interrupts and drivers elphel393/bootable-images/mmc/devicetree.dtb - mount rootfs from mmc partition elphel393/bootable-images/nand/devicetree.dtb - mount rootfs from NAND flash
uImage	bitbake linux-xlnx	kernel, drivers
rootfs.ubifs, rootfs.ubi, rootfs.tar.gz	bitbake core-image-elphel393	rootfs in different formats - .ubi for NAND, *.tar.gz for mmc

The file are located at elphel393/bootable-images/, which is a link to poky's deploy directory elphel393/poky/build/tmp/deploy/images/elphel393/

complete set of files for rootfs in NAND Flash: elphel393/bootable-images/nand/
complete set of files for rootfs on MMC (micro SD card): elphel393/bootable-images/mmc/

Write files to media and boot

Boot from micro SD card

Use recovery or regular μSD card
EXT4 partition mounted as /

Detailed instructions

Boot from NAND flash

The default boot option - power on.
A UBIFS image is written to /dev/mtd4 - is mounted as /

Detailed instructions

Notes

Read more about boot options.

Setup

Before building

Installing some dependencies:

sudo apt install git build-essential chrpath diffstat gawk texinfo zlib1g-dev python3-numpy libssl-dev

Installing old python2.7 (bitbake snapshot that we used has problem with python3.8):

sudo apt install python2.7
sudo ln -s python2.7 python2
sudo ln -s python2.7 python

Poky 2.7.1 Warrior (Kubuntu 20.04)

git clone -b warrior https://git.elphel.com/Elphel/elphel393.git

If you are a developer and plan to commit changes you should use ssh access instead: git clone -b warrior git@git.elphel.com:Elphel/elphel393.git

cd elphel393
./setup.py
cd poky
. ./oe-init-build-env
bitbake u-boot device-tree linux-xlnx core-image-elphel393

~~Poky 2.4 Rocko (Kubuntu 16.04)~~

~~git clone -b rocko https://git.elphel.com/Elphel/elphel393.git cd elphel393 ./setup.py cd poky . ./oe-init-build-env bitbake u-boot device-tree linux-xlnx core-image-elphel393~~

Notes (required actions are already listed above): When trying to run bitbake on a new install got problems with older bitbake version (current build uses specific snapshot) and python3.8. So I install old python2.7 and linked it:

sudo apt install python2.7 python-numpy
sudo ln -s python2.7 python2
sudo ln -s python2.7 python

After this bitbake just complained about missing tools (git, build-essential were already installed):

sudo apt install chrpath diffstat gawk makeinfo

Program test_mcntrl.py that builds x393.h header files from FPGA code that runs during ./setup.py requires numpy and results in error output if missing Still setup.py continues, you need to look through its output few lines after

* x393
git host: git.elphel.com
Clone and checkout: x393

You may install numpy with

sudo apt install python3-numpy

If you did not notice missing numpy, then during bitbake you will get error "build No rule to make target 'drivers/elphel/x393.o', needed by 'drivers/elphel/built-in.a'. Stop." in that case you need to install numpy, re-run setup.py (make sure numpy error is gone) then re-create linux kernel links during next build by erasing old ones:

bitbake linux-xlnx -c clean

and re-running

bitbake linux-xlnx device-tree core-image-elphel393

When building php: configure: error: zip support requires ZLIB. Use --with-zlib-dir=<DIR> to specify prefix where ZLIB include and library are located

sudo apt install zlib1g-dev

Compare built vs deployed software versions

cd elphel393
./check_versions.py root@192.168.0.9

check_versions.py output in terminal (web-393 revision on PC is ahead of the one installed on the camera)

Details for development

Development for 10393

Difference between revisions of "Poky manual"

Latest revision as of 11:22, 1 June 2022

Contents

Description

Use cases

Description

Features

Requirements

Output files

Write files to media and boot

Boot from micro SD card

Boot from NAND flash

Notes

Setup

Compare built vs deployed software versions

Details for development

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools

@@ Line 1: / Line 1: @@
+==<font color="blue">Description</font>==
+Build the firmware for Elphel 10393 series camera systems from scratch.
+Prebuilt releases can be found [https://community.elphel.com/files/393/ here].
+==<font color="blue">Use cases</font>==
+* Update Linux Kernel when developing a driver
+* Include application to the default firmware package
+* Update Elphel's PHP extension
+* C/C++ applications if developed on PC require a cross-compiler and require Yocto installation or some other alternative.
+Other projects written in Python, PHP, Javascript, HTML, C/C++ (gcc is installed on the system board) can be developed on the camera. Installing the framework is not required but it can simplify the development process.
 ==<font color="blue">Description</font>==
 * Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
+* Yocto Poky revision = 2.7.1 (Warrior) - <font color="green">builds in Kubuntu 20.04.</font> <font color="red">Older OSes are not supported. Roll back to get built in 16.04.</font>
+* <s>Yocto Poky revision = 2.4 (Rocko) - <font color="green">builds in Kubuntu 16.04.</font> <font color="red">Won't build in 18.04 or 20.04.</font></s>
+* <s>Yocto Poky revision = 2.0 (Jethro)</s>
-==<font color="blue">About</font>==
+==<font color="blue">Features</font>==
+* Environment is set up with a single script
+* All Elphel projects are configured for Eclipse IDE:
+** fpga projects (verilog)
+** kernel
+** applications projects
+* All steps for fpga projects are run from Eclipse IDE
+** Read about [http://blog.elphel.com/2016/05/tutorial-02-eclipse-based-fpga-development-environment-for-elphel-cameras/ setting up Eclipse based FPGA development]
+** [https://git.elphel.com/Elphel/vdt-plugin VDT Plugin repository]
-* Yocto Poky revision = 2.0
+* Once everything is built it supports updating software projects (files and binaries) on the target system
-* Host OS = Kubuntu 14.04.3 LTS x64
+** access is set up in yocto's ''local.conf'' (see ''setup.py'' output) + ''ssh-copy-id''
+** ''bitbake some-app -c target_scp'' from the command line or Eclipse IDE - see examples in '''elphel-apps-...''' at [https://git.elphel.com/Elphel Git]
-==<font color="blue">Required packages</font>==
+==<font color="blue">Requirements</font>==
-* Found in [http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html Yocto Project mega-manual]
+* Kubuntu 20.04 x64
+* <s>Kubuntu 18.04 x64 (as of Dec 2018) - Poky: 2.7.1 Warrior (not tested but might build w/o problems)</s>
+* <s>Kubuntu 16.04 x64 (as of Dec 2018) - Poky: 2.4 Rocko, 2.7.1 Warrior</s>
 ==<font color="blue">Output files</font>==
-Found in the poky's deploy directory:
+The names are listed as they appear in the u-boot configuration header file - actual output files have different names:
-* files for rootfs in NAND Flash: '''poky/build/tmp/deploy/images/elphel393/nand/'''
+{| class="wikitable"
-* files for rootfs on MMC (micro SD card): '''poky/build/tmp/deploy/images/elphel393/mmc/'''
+|-
+! File(s)
+! Build Command
+! Description
+|-
+| <font color='darkblue'>'''boot.bin'''</font>
+| ''bitbake u-boot''
+| u-boot as the first stage bootloader = Secondary Program Loader (SPL) that boots '''u-boot-dtb.img'''
+|-
+| <font color='darkblue'>'''u-boot-dtb.img'''</font>
+| ''bitbake u-boot''
+| full size u-boot with a stripped device tree (''cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img'')
+|-
+|valign='top'| <font color='darkblue'>'''devicetree.dtb'''</font>
+| bitbake device-tree
+| device tree with listed interfaces, zynq registers, interrupts and drivers
+* ''elphel393/bootable-images/mmc/devicetree.dtb'' - mount rootfs from mmc partition
+* ''elphel393/bootable-images/nand/devicetree.dtb'' - mount rootfs from NAND flash
+|-
+| <font color='darkblue'>'''uImage'''</font>
+| ''bitbake linux-xlnx''
+| kernel, drivers
+|-
+| <font color='darkblue'>'''rootfs.ubifs''', '''rootfs.ubi''', '''rootfs.tar.gz'''</font>
+| ''bitbake core-image-elphel393''
+| rootfs in different formats - *.ubi* for NAND, *.tar.gz for mmc
+|}
-These names are listed as they appear in the u-boot configuration header file - actual output files have different names:
+The file are located at '''elphel393/bootable-images/''', which is a link to poky's deploy directory '''elphel393/poky/build/tmp/deploy/images/elphel393/'''
-* '''boot.bin''' - u-boot as the first stage bootloader = Secondary Program Loader that boots '''u-boot-dtb.img'''
-* '''u-boot-dtb.img''' - full size u-boot with a stripped device tree (''cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img'')
-* '''devicetree.dtb''' - device tree with listed interfaces, zynq registers, interrupts and drivers
-* '''uImage''' - kernel, drivers
-* '''rootfs.ubi''' or '''rootfs.tar.gz''' - rootfs
-==<font color="blue">Boot options</font>==
+* complete set of files for rootfs in NAND Flash: ''elphel393/bootable-images/nand/''
+* complete set of files for rootfs on MMC (micro SD card): ''elphel393/bootable-images/mmc/''
+==<font color="blue">Write files to media and boot</font>==
 ===Boot from micro SD card===
-* The micro SD card/adapter must be modified for this boot mode (to keep CD pin high) - only then the camera will boot.
+* Use recovery or regular &mu;SD card
-* EXT4 partition mounted as '''/'''.
+* EXT4 partition mounted as '''/'''
-====Instructions====
+[[Sd_boot_rootfs|Detailed instructions]]
-PC:
-* After all of the targets are built use files from '''.../poky/build/tmp/deploy/images/elphel393/mmc/'''
-* Format to have 2 partitions: FAT32 and EXT4 (use gparted), the order of partitions is important or change bootargs in the device tree.
-* FAT32: copy the following files:
-*** '''boot.bin'''
-*** '''u-boot-dtb.img'''
-*** '''devicetree.dtb'''
-*** '''uImage'''
-* EXT4: mounted to some <mountpoint>
-** unpack the '''rootfs.tar.gz''', console:
- <font size='2'>tar -C <mountpoint> -xzpf rootfs.tar.gz</font>
-Camera:
-* Insert, power on
 ===Boot from NAND flash===
 * The default boot option - power on.
-* A UBIFS image of rootfs is written to /dev/mtd4 ('''ubi0:elphel393''' in bootargs in the device tree)
+* A UBIFS image is written to /dev/mtd4 - is mounted as '''/'''
-====Reflash factory image (update software/firmware or corrupt flash) ====
+[[NAND_flash_boot_rootfs|Detailed instructions]]
-* Use files from '''.../poky/build/tmp/deploy/images/elphel393/nand/'''
-* Copy all of the image files to a micro SD card (or boot first from the card then copy over network)
+===Notes===
-* Boot from micro SD card.
+* [[Boot_options_393|'''Read more''']] about boot options.
-* http://www.linux-mtd.infradead.org/faq/ubifs.html#L_mkfubifs
 ==<font color="blue">Setup</font>==
-* Step 1: Copy the following lines to <b>some_script.sh</b>, modify and run:
+* Before building
+Installing some dependencies:
+ sudo apt install git build-essential chrpath diffstat gawk texinfo zlib1g-dev python3-numpy libssl-dev
+Installing old python2.7 (bitbake snapshot that we used has problem with python3.8):
+ sudo apt install python2.7
+ sudo ln -s python2.7 python2
+ sudo ln -s python2.7 python
+* Poky 2.7.1 Warrior (Kubuntu 20.04)
 <font size='2'>
-  '''#Part 1'''
+  git clone -b '''warrior''' https://git.elphel.com/Elphel/elphel393.git
+If you are a developer and plan to commit changes you should use ssh access instead: git clone -b warrior git@git.elphel.com:Elphel/elphel393.git
- git clone -b master git://git.yoctoproject.org/poky.git poky
- cd poky; git checkout 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
+  cd elphel393
+  ./setup.py
-  git clone -b master https://github.com/Elphel/meta-ezynq.git meta-ezynq
+  cd poky
- cd meta-ezynq; git checkout b8b1cd8e6dae7cadb51e987f966a4ffbcd6c6543; cd ..
- git clone -b master https://github.com/Elphel/meta-elphel393.git meta-elphel393
-  cd meta-elphel393; git checkout c5237f99d94217bff8c9180bed93ee673c9590a1; cd ..
- git clone -b master https://github.com/Xilinx/meta-xilinx.git meta-xilinx
-  cd meta-xilinx; git checkout cc146d6c170f100eb2f445047969893faa7a6a55; cd ..
- git clone -b master git://git.openembedded.org/meta-openembedded meta-openembedded
- cd meta-openembedded; git checkout 73854a05565b30a5ca146ac53959c679b27815aa; cd ..
- '''#Part 2 : init environment and auto-fill Yocto's conf-files'''
- CONF_NOTES="meta/conf/conf-notes.txt"
- if [ -f $CONF_NOTES ]; then
-     rm $CONF_NOTES
- fi
- echo "Common \"elphel393\" targets are:" >> $CONF_NOTES
- echo "    u-boot" >> $CONF_NOTES
- echo "    device-tree" >> $CONF_NOTES
- echo "    linux-xlnx" >> $CONF_NOTES
- echo "    core-image-elphel393" >> $CONF_NOTES
- CURRENT_PATH=$(dirname $(readlink -f "$0"))
   . ./oe-init-build-env
+  bitbake u-boot device-tree linux-xlnx core-image-elphel393
- BBLAYERS_CONF="conf/bblayers.conf"
+</font>
- echo "BBLAYERS = \" \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-yocto \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-yocto-bsp \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-ezynq \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-elphel393 \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-xilinx \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-openembedded/meta-oe \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-openembedded/meta-python \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-openembedded/meta-networking \\" >> $BBLAYERS_CONF
- echo "  $CURRENT_PATH/meta-openembedded/meta-webserver \\" >> $BBLAYERS_CONF
- echo "  \"" >> $BBLAYERS_CONF
- LOCAL_CONF="conf/local.conf"
- # <font color='red'>change the MACHINE</font>
- echo "MACHINE ?= \"'''elphel393'''\"" >> $LOCAL_CONF
- # <font color='red'>Elphel's MIRROR website, '''\n''' is important</font>
- echo "MIRRORS =+ \"http://.*/.*     http://mirror.elphel.com/elphel393_mirror/ \n \"" >> $LOCAL_CONF
-* Step 2: re-init environment
+* <s>Poky 2.4 Rocko (Kubuntu 16.04)
+<font size='2'>
+ git clone -b '''rocko''' https://git.elphel.com/Elphel/elphel393.git
+ cd elphel393
+ ./setup.py
   cd poky
   . ./oe-init-build-env
+ bitbake u-boot device-tree linux-xlnx core-image-elphel393
+<!--''Temporary: linux-xlnx in Rocko depends on branch 'lwir' for x393. Change in projects.json line 42 from 'master' to 'lwir'.''-->
+</font></s>
+* Notes (required actions are already listed above): When trying to run bitbake on a new install got problems with older bitbake version (current build uses specific snapshot) and python3.8. So I install old python2.7 and linked it:
+ sudo apt install python2.7 python-numpy
+ sudo ln -s python2.7 python2
+ sudo ln -s python2.7 python
-* Step 3: build bootable image (3 targets)
+After this bitbake just complained about missing tools (git, build-essential were already installed):
- bitbake u-boot-ezynq device-tree linux-xlnx core-image-elphel393
-</font>
-==<font color="blue">Build bootloader</font>==
+  sudo apt install chrpath diffstat gawk makeinfo
-<font size='2'>
-  bitbake u-boot-ezynq
- or
- bitbake u-boot
- or
- bitbake virtual/bootloader
-</font>
-====Output====
-* poky/build/tmp/deploy/images/<machine>/'''boot.bin'''
-* poky/build/tmp/deploy/images/<machine>/'''u-boot-dtb.img'''
-====Notes====
+Program test_mcntrl.py that builds x393.h header files from FPGA code that runs during ./setup.py requires numpy and results in error output if missing Still setup.py continues, you need to look through its output few lines after
-* Since Poky 1.5.1 switched from u-boot-xlnx to u-boot - some of the u-boot-xlnx reached the upstream
+ * x393
-* Since Poky 1.5.1 u-boot introduced Kconfig and currently (as of 2016/01/22) is still migrating from autoconf - keeping both build methods working together.
+ git host: git.elphel.com
-* Current u-boot uses the board's device tree blob to make a stripped version for itself - uncompiled device tree files are present in the u-boot sources
+ Clone and checkout: x393
-==<font color="blue">Compile devicetree</font>==
+You may install numpy with
-<font size='2'>
+  sudo apt install python3-numpy
-  bitbake device-tree
-</font>
-====Output====
-* poky/build/tmp/deploy/images/<machine>/'''devicetree.dtb'''
-==<font color="blue">Build kernel</font>==
+If you did not notice missing numpy, then during bitbake you will get error "build No rule to make target 'drivers/elphel/x393.o', needed by 'drivers/elphel/built-in.a'.  Stop." in that case you need to install numpy, re-run setup.py (make sure numpy error is gone) then re-create linux kernel links during next build by erasing old ones:
-<font size='2'>
+  bitbake linux-xlnx -c clean
-  bitbake linux-xlnx
+and re-running
- or
+  bitbake linux-xlnx device-tree core-image-elphel393
-  bitbake virtual/kernel
-</font>
-====Output====
-* poky/build/tmp/deploy/images/<machine>/'''uImage''' (kernel)
-====Notes====
+When building php: configure: error: zip support requires ZLIB. Use --with-zlib-dir=<DIR> to specify prefix where ZLIB include and library are located
-* Kernel version is 4.0, linux-xlnx
+ sudo apt install zlib1g-dev
-* Device Tree Blob is compiled separately in linux-xlnx
-* Uncompiled device tree files are present in the linux sources provided by linux-xlnx
-==<font color="blue">Build ramdisk.image.gz</font>==
+==<font color="blue">Compare '''built''' vs '''deployed''' software versions</font>==
 <font size='2'>
-  bitbake core-image-elphel393
+  cd elphel393
+ ./check_versions.py root@192.168.0.9
 </font>
-====Output====
-* poky/build/tmp/deploy/images/<machine>/'''uramdisk.image.gz''' (the file has *.u-boot extension after it's built)
-==<font color="blue">Notes</font>==
+{|
-* <span style='color:green'>[FIXED]</span> lighttpd-1.4.39, mod-cgi
+|[[File:Check versions.png|thumb|400px|check_versions.py output in terminal (web-393 revision on PC is ahead of the one installed on the camera)]]
-compile warning:
+|}
- implicit declaration of function 'chunkqueue_written'
-results in lighttpd failing to start if mod-cgi is enabled:
- Starting Lighttpd Web Server: 2016-01-26 18:42:17: (/data/GIT_fresh/poky/build/tmp/work/cortexa9-vfp-neon-poky-linux-gnueabi/lighttpd/1.4.36-r0/lighttpd-1.4.36/src/plugin.c.169) dlopen() failed for: /usr/lib/mod_cgi.so /usr/lib/mod_cgi.so: undefined symbol: chunkqueue_w
--01-26 18:42:17: (/data/GIT_fresh/poky/build/tmp/work/cortexa9-vfp-neon-poky-linux-gnueabi/lighttpd/1.4.36-r0/lighttpd-1.4.36/src/server.c.679) loading plugins finally failed
- lighttpd.
-Note: why 1.4.36?!!
-<b>solution</b>: exclude the '''0001-mod_cgi-buffers-data-without-bound.patch''' from the file list that comes with the '''poky/meta/recipes-extended/lighttpd_1.4.39.bb'''
+==<font color="blue">Details for development</font>==
+[[Development_for_10393|'''Development for 10393''']]