Difference between revisions of "Poky manual"

From ElphelWiki
Jump to: navigation, search
(Build ramdisk.image.gz)
(Setup)
 
(171 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
==<font color="blue">Description</font>==
 +
Build the firmware for Elphel 10393 series camera systems from scratch.
 +
 +
Prebuilt releases can be found [https://community.elphel.com/files/393/ here].
 +
 +
==<font color="blue">Use cases</font>==
 +
* Update Linux Kernel when developing a driver
 +
* Include application to the default firmware package
 +
* Update Elphel's PHP extension
 +
* C/C++ applications if developed on PC require a cross-compiler and require Yocto installation or some other alternative.
 +
 +
Other projects written in Python, PHP, Javascript, HTML, C/C++ (gcc is installed on the system board) can be developed on the camera. Installing the framework is not required but it can simplify the development process.
 +
 
==<font color="blue">Description</font>==
 
==<font color="blue">Description</font>==
  
 
* Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
 
* Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
 +
* Yocto Poky revision = 2.7.1 (Warrior) - <font color="green">builds in Kubuntu 20.04.</font> <font color="red">Older OSes are not supported. Roll back to get built in 16.04.</font>
 +
* <s>Yocto Poky revision = 2.4 (Rocko) - <font color="green">builds in Kubuntu 16.04.</font> <font color="red">Won't build in 18.04 or 20.04.</font></s>
 +
* <s>Yocto Poky revision = 2.0 (Jethro)</s>
  
==<font color="blue">About</font>==
+
==<font color="blue">Features</font>==
 +
* Environment is set up with a single script
 +
* All Elphel projects are configured for Eclipse IDE:
 +
** fpga projects (verilog)
 +
** kernel
 +
** applications projects
 +
* All steps for fpga projects are run from Eclipse IDE
 +
** Read about [http://blog.elphel.com/2016/05/tutorial-02-eclipse-based-fpga-development-environment-for-elphel-cameras/ setting up Eclipse based FPGA development]
 +
** [https://git.elphel.com/Elphel/vdt-plugin VDT Plugin repository]
  
* Yocto Poky revision = 2.0
+
* Once everything is built it supports updating software projects (files and binaries) on the target system
* Host OS = Kubuntu 14.04.3 LTS x64
+
** access is set up in yocto's ''local.conf'' (see ''setup.py'' output) + ''ssh-copy-id''
 +
** ''bitbake some-app -c target_scp'' from the command line or Eclipse IDE - see examples in '''elphel-apps-...''' at [https://git.elphel.com/Elphel Git]
  
==<font color="blue">Required packages</font>==
+
==<font color="blue">Requirements</font>==
* Found in [http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html Yocto Project mega-manual]
+
* Kubuntu 20.04 x64
 +
* <s>Kubuntu 18.04 x64 (as of Dec 2018) - Poky: 2.7.1 Warrior (not tested but might build w/o problems)</s>
 +
* <s>Kubuntu 16.04 x64 (as of Dec 2018) - Poky: 2.4 Rocko, 2.7.1 Warrior</s>
  
 
==<font color="blue">Output files</font>==
 
==<font color="blue">Output files</font>==
Found in the poky's deploy directory: '''poky/build/tmp/deploy/images/'''
+
The names are listed as they appear in the u-boot configuration header file - actual output files have different names:
 +
{| class="wikitable"
 +
|-
 +
! File(s)
 +
! Build Command
 +
! Description
 +
|-
 +
| <font color='darkblue'>'''boot.bin'''</font>
 +
| ''bitbake u-boot''
 +
| u-boot as the first stage bootloader = Secondary Program Loader (SPL) that boots '''u-boot-dtb.img'''
 +
|-
 +
| <font color='darkblue'>'''u-boot-dtb.img'''</font>
 +
| ''bitbake u-boot''
 +
| full size u-boot with a stripped device tree (''cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img'')
 +
|-
 +
|valign='top'| <font color='darkblue'>'''devicetree.dtb'''</font>
 +
| bitbake device-tree
 +
| device tree with listed interfaces, zynq registers, interrupts and drivers
 +
* ''elphel393/bootable-images/mmc/devicetree.dtb'' - mount rootfs from mmc partition
 +
* ''elphel393/bootable-images/nand/devicetree.dtb'' - mount rootfs from NAND flash
 +
|-
 +
| <font color='darkblue'>'''uImage'''</font>
 +
| ''bitbake linux-xlnx''
 +
| kernel, drivers
 +
|-
 +
| <font color='darkblue'>'''rootfs.ubifs''', '''rootfs.ubi''', '''rootfs.tar.gz'''</font>
 +
| ''bitbake core-image-elphel393''
 +
| rootfs in different formats - *.ubi* for NAND, *.tar.gz for mmc
 +
|}
 +
 
 +
The file are located at '''elphel393/bootable-images/''', which is a link to poky's deploy directory '''elphel393/poky/build/tmp/deploy/images/elphel393/'''
 +
 
 +
* complete set of files for rootfs in NAND Flash: ''elphel393/bootable-images/nand/''
 +
* complete set of files for rootfs on MMC (micro SD card): ''elphel393/bootable-images/mmc/''
  
These names are listed as they appear in the u-boot configuration header file - actual output files have different names:
+
==<font color="blue">Write files to media and boot</font>==
* '''boot.bin'''+('''u-boot-dtb.img''') - u-boot as the first stage bootloader = Secondary Program Loader that boots '''u-boot-dtb.img'''
+
===Boot from micro SD card===
* '''devicetree.dtb''' - device tree with described interfaces, zynq registers, interrupts and drivers
+
* Use recovery or regular &mu;SD card
* '''uImage''' - kernel, drivers
+
* EXT4 partition mounted as '''/'''
* '''uramdisk.image.gz''' - applications
+
[[Sd_boot_rootfs|Detailed instructions]]
  
Copy them on the micro SD card (FAT partition) > run '''boot''' once in the u-boot command line.
+
===Boot from NAND flash===
 +
* The default boot option - power on.
 +
* A UBIFS image is written to /dev/mtd4 - is mounted as '''/'''
 +
[[NAND_flash_boot_rootfs|Detailed instructions]]
  
==<font color="blue">Boot options</font>==
+
===Notes===
* Auto unpacking the root file system image to RAM when booting:
+
* [[Boot_options_393|'''Read more''']] about boot options.
** Check u-boot default environment variables when building the bootloader.
 
** Keep '''uramdisk.image.gz''' along with all the other files on the FAT partition of (micro)SD card.
 
* <font style='color:rgba(100,100,100,1)'>Root file system on EXT2(EXT3,etc.) partition of (micro)SD card.</font>
 
  
 
==<font color="blue">Setup</font>==
 
==<font color="blue">Setup</font>==
* Step 1: Copy the following lines to <b>some_script.sh</b>, modify and run:
+
* Before building
 +
Installing some dependencies:
 +
sudo apt install git build-essential chrpath diffstat gawk texinfo zlib1g-dev python3-numpy libssl-dev
 +
Installing old python2.7 (bitbake snapshot that we used has problem with python3.8):
 +
sudo apt install python2.7
 +
sudo ln -s python2.7 python2
 +
sudo ln -s python2.7 python
 +
 
 +
* Poky 2.7.1 Warrior (Kubuntu 20.04)
 
<font size='2'>
 
<font size='2'>
  '''#Part 1'''
+
  git clone -b '''warrior''' https://git.elphel.com/Elphel/elphel393.git
 +
 
 +
If you are a developer and plan to commit changes you should use ssh access instead: git clone -b warrior git@git.elphel.com:Elphel/elphel393.git
 
   
 
   
git clone -b master git://git.yoctoproject.org/poky.git poky
+
 
cd poky; git checkout 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
+
  cd elphel393
 
+
  ./setup.py
git clone -b master https://github.com/Elphel/meta-ezynq.git meta-ezynq
+
  cd poky
  cd meta-ezynq; git checkout 8818b2b560245777a730d6715e2be34c9c6545bc; cd ..
 
 
git clone -b master https://github.com/Elphel/meta-elphel393.git meta-elphel393
 
  cd meta-elphel393; git checkout fceb0b9649186eced2e4f2a1cab7d3457de3e526; cd ..
 
 
git clone -b master https://github.com/Xilinx/meta-xilinx.git meta-xilinx
 
  cd meta-xilinx; git checkout cc146d6c170f100eb2f445047969893faa7a6a55; cd ..
 
 
git clone -b master git://git.openembedded.org/meta-openembedded meta-openembedded
 
cd meta-openembedded; git checkout 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c; cd ..
 
 
'''#Part 2 : init environment and auto-fill Yocto's conf-files'''
 
 
CURRENT_PATH=$(dirname $(readlink -f "$0"))
 
 
  . ./oe-init-build-env
 
  . ./oe-init-build-env
   
+
  bitbake u-boot device-tree linux-xlnx core-image-elphel393
BBLAYERS_CONF="conf/bblayers.conf"
+
</font>
 
echo "BBLAYERS = \" \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-yocto \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-yocto-bsp \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-ezynq \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-elphel393 \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-xilinx \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-openembedded/meta-oe \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-openembedded/meta-python \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-openembedded/meta-networking \\" >> $BBLAYERS_CONF
 
echo "  $CURRENT_PATH/meta-openembedded/meta-webserver \\" >> $BBLAYERS_CONF
 
echo "  \"" >> $BBLAYERS_CONF
 
 
LOCAL_CONF="conf/local.conf"
 
 
# <font color='red'>change the MACHINE</font>
 
echo "MACHINE ?= \"'''elphel393'''\"" >> $LOCAL_CONF
 
# <font color='red'>Elphel's MIRROR website, '''\n''' is important</font>
 
echo "MIRRORS =+ \"http://.*/.*    http://mirror.elphel.com/elphel393_mirror/ \n \"" >> $LOCAL_CONF
 
  
* Step 2: re-init environment
+
* <s>Poky 2.4 Rocko (Kubuntu 16.04)
 +
<font size='2'>
 +
git clone -b '''rocko''' https://git.elphel.com/Elphel/elphel393.git
 +
cd elphel393
 +
./setup.py
 
  cd poky
 
  cd poky
 
  . ./oe-init-build-env
 
  . ./oe-init-build-env
 +
bitbake u-boot device-tree linux-xlnx core-image-elphel393
 +
<!--''Temporary: linux-xlnx in Rocko depends on branch 'lwir' for x393. Change in projects.json line 42 from 'master' to 'lwir'.''-->
 +
</font></s>
 +
 +
* Notes (required actions are already listed above): When trying to run bitbake on a new install got problems with older bitbake version (current build uses specific snapshot) and python3.8. So I install old python2.7 and linked it:
 +
sudo apt install python2.7 python-numpy
 +
sudo ln -s python2.7 python2
 +
sudo ln -s python2.7 python
 +
 +
After this bitbake just complained about missing tools (git, build-essential were already installed):
  
* Step 3: build bootable image (3 targets)
+
sudo apt install chrpath diffstat gawk makeinfo
  bitbake u-boot-ezynq device-tree linux-xlnx core-image-elphel393
+
 
</font>
+
Program test_mcntrl.py that builds x393.h header files from FPGA code that runs during ./setup.py requires numpy and results in error output if missing Still setup.py continues, you need to look through its output few lines after
 +
* x393
 +
git host: git.elphel.com
 +
  Clone and checkout: x393
  
==<font color="blue">Build bootloader</font>==
+
You may install numpy with
<font size='2'>
+
  sudo apt install python3-numpy
  bitbake u-boot-ezynq
 
or
 
bitbake u-boot
 
or
 
bitbake virtual/bootloader
 
</font>
 
====Output====
 
* poky/build/tmp/deploy/images/<machine>/'''boot.bin'''
 
* poky/build/tmp/deploy/images/<machine>/'''u-boot-dtb.img'''
 
  
====Notes====
+
If you did not notice missing numpy, then during bitbake you will get error "build No rule to make target 'drivers/elphel/x393.o', needed by 'drivers/elphel/built-in.a'. Stop." in that case you need to install numpy, re-run setup.py (make sure numpy error is gone) then re-create linux kernel links during next build by erasing old ones:
* Since Poky 1.5.1 switched from u-boot-xlnx to u-boot - some of the u-boot-xlnx reached the upstream
+
bitbake linux-xlnx -c clean
* Since Poky 1.5.1 u-boot introduced Kconfig and currently (as of 2016/01/22) is still migrating from autoconf - keeping both build methods working together.
+
and re-running
* Current u-boot uses the board's device tree blob to make a stripped version for itself - uncompiled device tree files are present in the u-boot sources
+
bitbake linux-xlnx device-tree core-image-elphel393
  
==<font color="blue">Compile devicetree</font>==
+
When building php: configure: error: zip support requires ZLIB. Use --with-zlib-dir=<DIR> to specify prefix where ZLIB include and library are located
<font size='2'>
+
  sudo apt install zlib1g-dev
  bitbake device-tree
 
</font>
 
====Output====
 
* poky/build/tmp/deploy/images/<machine>/'''devicetree.dtb'''
 
  
==<font color="blue">Build kernel</font>==
+
==<font color="blue">Compare '''built''' vs '''deployed''' software versions</font>==
 
<font size='2'>
 
<font size='2'>
  bitbake linux-xlnx
+
  cd elphel393
  or
+
  ./check_versions.py root@192.168.0.9
bitbake virtual/kernel
 
 
</font>
 
</font>
====Output====
 
* poky/build/tmp/deploy/images/<machine>/'''uImage''' (kernel)
 
  
====Notes====
+
{|
* Kernel version is 4.0, linux-xlnx
+
|[[File:Check versions.png|thumb|400px|check_versions.py output in terminal (web-393 revision on PC is ahead of the one installed on the camera)]]
* Device Tree Blob is compiled separately in linux-xlnx
+
|}
* Uncompiled device tree files are present in the linux sources provided by linux-xlnx
 
  
==<font color="blue">Build ramdisk.image.gz</font>==
+
==<font color="blue">Details for development</font>==
<font size='2'>
+
[[Development_for_10393|'''Development for 10393''']]
bitbake core-image-elphel393
 
</font>
 
====Output====
 
* poky/build/tmp/deploy/images/<machine>/'''uramdisk.image.gz''' (the file has *.u-boot extension after it's built)
 

Latest revision as of 12:22, 1 June 2022

Description

Build the firmware for Elphel 10393 series camera systems from scratch.

Prebuilt releases can be found here.

Use cases

  • Update Linux Kernel when developing a driver
  • Include application to the default firmware package
  • Update Elphel's PHP extension
  • C/C++ applications if developed on PC require a cross-compiler and require Yocto installation or some other alternative.

Other projects written in Python, PHP, Javascript, HTML, C/C++ (gcc is installed on the system board) can be developed on the camera. Installing the framework is not required but it can simplify the development process.

Description

  • Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
  • Yocto Poky revision = 2.7.1 (Warrior) - builds in Kubuntu 20.04. Older OSes are not supported. Roll back to get built in 16.04.
  • Yocto Poky revision = 2.4 (Rocko) - builds in Kubuntu 16.04. Won't build in 18.04 or 20.04.
  • Yocto Poky revision = 2.0 (Jethro)

Features

  • Once everything is built it supports updating software projects (files and binaries) on the target system
    • access is set up in yocto's local.conf (see setup.py output) + ssh-copy-id
    • bitbake some-app -c target_scp from the command line or Eclipse IDE - see examples in elphel-apps-... at Git

Requirements

  • Kubuntu 20.04 x64
  • Kubuntu 18.04 x64 (as of Dec 2018) - Poky: 2.7.1 Warrior (not tested but might build w/o problems)
  • Kubuntu 16.04 x64 (as of Dec 2018) - Poky: 2.4 Rocko, 2.7.1 Warrior

Output files

The names are listed as they appear in the u-boot configuration header file - actual output files have different names:

File(s) Build Command Description
boot.bin bitbake u-boot u-boot as the first stage bootloader = Secondary Program Loader (SPL) that boots u-boot-dtb.img
u-boot-dtb.img bitbake u-boot full size u-boot with a stripped device tree (cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img)
devicetree.dtb bitbake device-tree device tree with listed interfaces, zynq registers, interrupts and drivers
  • elphel393/bootable-images/mmc/devicetree.dtb - mount rootfs from mmc partition
  • elphel393/bootable-images/nand/devicetree.dtb - mount rootfs from NAND flash
uImage bitbake linux-xlnx kernel, drivers
rootfs.ubifs, rootfs.ubi, rootfs.tar.gz bitbake core-image-elphel393 rootfs in different formats - *.ubi* for NAND, *.tar.gz for mmc

The file are located at elphel393/bootable-images/, which is a link to poky's deploy directory elphel393/poky/build/tmp/deploy/images/elphel393/

  • complete set of files for rootfs in NAND Flash: elphel393/bootable-images/nand/
  • complete set of files for rootfs on MMC (micro SD card): elphel393/bootable-images/mmc/

Write files to media and boot

Boot from micro SD card

  • Use recovery or regular μSD card
  • EXT4 partition mounted as /

Detailed instructions

Boot from NAND flash

  • The default boot option - power on.
  • A UBIFS image is written to /dev/mtd4 - is mounted as /

Detailed instructions

Notes

Setup

  • Before building

Installing some dependencies:

sudo apt install git build-essential chrpath diffstat gawk texinfo zlib1g-dev python3-numpy libssl-dev

Installing old python2.7 (bitbake snapshot that we used has problem with python3.8):

sudo apt install python2.7
sudo ln -s python2.7 python2
sudo ln -s python2.7 python
  • Poky 2.7.1 Warrior (Kubuntu 20.04)

git clone -b warrior https://git.elphel.com/Elphel/elphel393.git

If you are a developer and plan to commit changes you should use ssh access instead: git clone -b warrior git@git.elphel.com:Elphel/elphel393.git


cd elphel393
./setup.py
cd poky
. ./oe-init-build-env
bitbake u-boot device-tree linux-xlnx core-image-elphel393

  • Poky 2.4 Rocko (Kubuntu 16.04)

git clone -b rocko https://git.elphel.com/Elphel/elphel393.git
cd elphel393
./setup.py
cd poky
. ./oe-init-build-env
bitbake u-boot device-tree linux-xlnx core-image-elphel393

  • Notes (required actions are already listed above): When trying to run bitbake on a new install got problems with older bitbake version (current build uses specific snapshot) and python3.8. So I install old python2.7 and linked it:
sudo apt install python2.7 python-numpy
sudo ln -s python2.7 python2
sudo ln -s python2.7 python

After this bitbake just complained about missing tools (git, build-essential were already installed):

sudo apt install chrpath diffstat gawk makeinfo

Program test_mcntrl.py that builds x393.h header files from FPGA code that runs during ./setup.py requires numpy and results in error output if missing Still setup.py continues, you need to look through its output few lines after

* x393
git host: git.elphel.com
Clone and checkout: x393

You may install numpy with

sudo apt install python3-numpy

If you did not notice missing numpy, then during bitbake you will get error "build No rule to make target 'drivers/elphel/x393.o', needed by 'drivers/elphel/built-in.a'. Stop." in that case you need to install numpy, re-run setup.py (make sure numpy error is gone) then re-create linux kernel links during next build by erasing old ones:

bitbake linux-xlnx -c clean

and re-running

bitbake linux-xlnx device-tree core-image-elphel393

When building php: configure: error: zip support requires ZLIB. Use --with-zlib-dir=<DIR> to specify prefix where ZLIB include and library are located

sudo apt install zlib1g-dev

Compare built vs deployed software versions

cd elphel393
./check_versions.py root@192.168.0.9

check_versions.py output in terminal (web-393 revision on PC is ahead of the one installed on the camera)

Details for development

Development for 10393