Difference between revisions of "Poky manual"

From ElphelWiki
Jump to: navigation, search
(Required packages)
(Setup)
 
(134 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
==<font color="blue">Description</font>==
 
==<font color="blue">Description</font>==
 +
Build the firmware for Elphel 10393 series camera systems from scratch.
  
* Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
+
Prebuilt releases can be found [https://community.elphel.com/files/393/ here].
  
==<font color="blue">About</font>==
+
==<font color="blue">Use cases</font>==
 +
* Update Linux Kernel when developing a driver
 +
* Include application to the default firmware package
 +
* Update Elphel's PHP extension
 +
* C/C++ applications if developed on PC require a cross-compiler and require Yocto installation or some other alternative.
  
* Yocto Poky revision = 2.0
+
Other projects written in Python, PHP, Javascript, HTML, C/C++ (gcc is installed on the system board) can be developed on the camera. Installing the framework is not required but it can simplify the development process.
* Host OS = Kubuntu 14.04.3 LTS x64
 
  
==<font color="blue">Required packages</font>==
+
==<font color="blue">Description</font>==
* Found in [http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html Yocto Project mega-manual] :
 
Ubuntu and Debian
 
  
    sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib \
+
* Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
    build-essential chrpath socat libsdl1.2-dev xterm
+
* Yocto Poky revision = 2.7.1 (Warrior) - <font color="green">builds in Kubuntu 20.04.</font> <font color="red">Older OSes are not supported. Roll back to get built in 16.04.</font>
                       
+
* <s>Yocto Poky revision = 2.4 (Rocko) - <font color="green">builds in Kubuntu 16.04.</font> <font color="red">Won't build in 18.04 or 20.04.</font></s>
 +
* <s>Yocto Poky revision = 2.0 (Jethro)</s>
  
Fedora
+
==<font color="blue">Features</font>==
 +
* Environment is set up with a single script
 +
* All Elphel projects are configured for Eclipse IDE:
 +
** fpga projects (verilog)
 +
** kernel
 +
** applications projects
 +
* All steps for fpga projects are run from Eclipse IDE
 +
** Read about [http://blog.elphel.com/2016/05/tutorial-02-eclipse-based-fpga-development-environment-for-elphel-cameras/ setting up Eclipse based FPGA development]
 +
** [https://git.elphel.com/Elphel/vdt-plugin VDT Plugin repository]
  
    sudo dnf install gawk make wget tar bzip2 gzip python unzip perl patch \
+
* Once everything is built it supports updating software projects (files and binaries) on the target system
    diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \
+
** access is set up in yocto's ''local.conf'' (see ''setup.py'' output) + ''ssh-copy-id''
    ccache perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue socat \
+
** ''bitbake some-app -c target_scp'' from the command line or Eclipse IDE - see examples in '''elphel-apps-...''' at [https://git.elphel.com/Elphel Git]
    findutils which SDL-devel xterm
 
                       
 
  
OpenSUSE
+
==<font color="blue">Requirements</font>==
 +
* Kubuntu 20.04 x64
 +
* <s>Kubuntu 18.04 x64 (as of Dec 2018) - Poky: 2.7.1 Warrior (not tested but might build w/o problems)</s>
 +
* <s>Kubuntu 16.04 x64 (as of Dec 2018) - Poky: 2.4 Rocko, 2.7.1 Warrior</s>
  
    sudo zypper install python gcc gcc-c++ git chrpath make wget python-xml \
+
==<font color="blue">Output files</font>==
    diffstat makeinfo python-curses patch socat libSDL-devel xterm
+
The names are listed as they appear in the u-boot configuration header file - actual output files have different names:
                       
+
{| class="wikitable"
 
+
|-
CentOS
+
! File(s)
 
+
! Build Command
    sudo yum install gawk make wget tar bzip2 gzip python unzip perl patch \
+
! Description
    diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \
+
|-
    perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue SDL-devel xterm
+
| <font color='darkblue'>'''boot.bin'''</font>
 +
| ''bitbake u-boot''
 +
| u-boot as the first stage bootloader = Secondary Program Loader (SPL) that boots '''u-boot-dtb.img'''
 +
|-
 +
| <font color='darkblue'>'''u-boot-dtb.img'''</font>
 +
| ''bitbake u-boot''
 +
| full size u-boot with a stripped device tree (''cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img'')
 +
|-
 +
|valign='top'| <font color='darkblue'>'''devicetree.dtb'''</font>
 +
| bitbake device-tree
 +
| device tree with listed interfaces, zynq registers, interrupts and drivers
 +
* ''elphel393/bootable-images/mmc/devicetree.dtb'' - mount rootfs from mmc partition
 +
* ''elphel393/bootable-images/nand/devicetree.dtb'' - mount rootfs from NAND flash
 +
|-
 +
| <font color='darkblue'>'''uImage'''</font>
 +
| ''bitbake linux-xlnx''
 +
| kernel, drivers
 +
|-
 +
| <font color='darkblue'>'''rootfs.ubifs''', '''rootfs.ubi''', '''rootfs.tar.gz'''</font>
 +
| ''bitbake core-image-elphel393''
 +
| rootfs in different formats - *.ubi* for NAND, *.tar.gz for mmc
 +
|}
  
==<font color="blue">Output files</font>==
+
The file are located at '''elphel393/bootable-images/''', which is a link to poky's deploy directory '''elphel393/poky/build/tmp/deploy/images/elphel393/'''
Found in the poky's deploy directory:
 
* files for rootfs in NAND Flash: '''poky/build/tmp/deploy/images/elphel393/nand/'''
 
* files for rootfs on MMC (micro SD card): '''poky/build/tmp/deploy/images/elphel393/mmc/'''
 
  
These names are listed as they appear in the u-boot configuration header file - actual output files have different names:
+
* complete set of files for rootfs in NAND Flash: ''elphel393/bootable-images/nand/''
* '''boot.bin''' - u-boot as the first stage bootloader = Secondary Program Loader that boots '''u-boot-dtb.img'''
+
* complete set of files for rootfs on MMC (micro SD card): ''elphel393/bootable-images/mmc/''
* '''u-boot-dtb.img''' - full size u-boot with a stripped device tree (''cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img'')
 
* '''devicetree.dtb''' - device tree with listed interfaces, zynq registers, interrupts and drivers
 
* '''uImage''' - kernel, drivers
 
* '''rootfs.ubifs''' or '''rootfs.tar.gz''' - rootfs
 
  
==<font color="blue">Boot options</font>==
+
==<font color="blue">Write files to media and boot</font>==
===Boot from micro SD card (rootfs)===
+
===Boot from micro SD card===
* The micro SD card/adapter must be modified for this boot mode (to keep CD pin high) - only then the camera will boot
+
* Use recovery or regular &mu;SD card
 
* EXT4 partition mounted as '''/'''
 
* EXT4 partition mounted as '''/'''
* [[Sd_boot_rootfs|Detailed instructions]]
+
[[Sd_boot_rootfs|Detailed instructions]]
  
===Boot from NAND flash (rootfs)===
+
===Boot from NAND flash===
 
* The default boot option - power on.
 
* The default boot option - power on.
 
* A UBIFS image is written to /dev/mtd4 - is mounted as '''/'''
 
* A UBIFS image is written to /dev/mtd4 - is mounted as '''/'''
* [[NAND_flash_boot_rootfs|Detailed instructions]]
+
[[NAND_flash_boot_rootfs|Detailed instructions]]
  
 
===Notes===
 
===Notes===
* It is possible to unpack rootfs to RAM not mounting any partitions ('''/''' in RAM, ramdisk):
+
* [[Boot_options_393|'''Read more''']] about boot options.
** Changes are not stored
 
** Boot faster?
 
** Work faster?
 
** Modify env commands in the u-boot (in elphel393.h or when in the u-boot command line)
 
** Carefully check the actual and hardcoded images' sizes in the u-boot (elphel393.h) and the device tree (elphel393_xxx.dts) with the actual sizes.
 
  
 
==<font color="blue">Setup</font>==
 
==<font color="blue">Setup</font>==
*  
+
* Before building
 +
Installing some dependencies:
 +
sudo apt install git build-essential chrpath diffstat gawk texinfo zlib1g-dev python3-numpy libssl-dev
 +
Installing old python2.7 (bitbake snapshot that we used has problem with python3.8):
 +
sudo apt install python2.7
 +
sudo ln -s python2.7 python2
 +
sudo ln -s python2.7 python
 +
 
 +
* Poky 2.7.1 Warrior (Kubuntu 20.04)
 
<font size='2'>
 
<font size='2'>
  $ git clone https://github.com/Elphel/elphel393.git
+
  git clone -b '''warrior''' https://git.elphel.com/Elphel/elphel393.git
  $ cd elphel393
+
 
  $ ./setup.sh
+
If you are a developer and plan to commit changes you should use ssh access instead: git clone -b warrior git@git.elphel.com:Elphel/elphel393.git
  $ cd poky
+
  $ . ./oe-init-build-env
+
 
  $ bitbake u-boot device-tree linux-xlnx core-image-elphel393
+
  cd elphel393
 +
  ./setup.py
 +
  cd poky
 +
  . ./oe-init-build-env
 +
  bitbake u-boot device-tree linux-xlnx core-image-elphel393
 
</font>
 
</font>
  
==<font color="blue">Build bootloader</font>==
+
* <s>Poky 2.4 Rocko (Kubuntu 16.04)
 
<font size='2'>
 
<font size='2'>
  bitbake u-boot-ezynq
+
  git clone -b '''rocko''' https://git.elphel.com/Elphel/elphel393.git
  or
+
  cd elphel393
  bitbake u-boot
+
  ./setup.py
  or
+
  cd poky
  bitbake virtual/bootloader
+
  . ./oe-init-build-env
</font>
+
bitbake u-boot device-tree linux-xlnx core-image-elphel393
====Output====
+
<!--''Temporary: linux-xlnx in Rocko depends on branch 'lwir' for x393. Change in projects.json line 42 from 'master' to 'lwir'.''-->
* poky/build/tmp/deploy/images/<machine>/'''boot.bin'''
+
</font></s>
* poky/build/tmp/deploy/images/<machine>/'''u-boot-dtb.img'''
+
 
 +
* Notes (required actions are already listed above): When trying to run bitbake on a new install got problems with older bitbake version (current build uses specific snapshot) and python3.8. So I install old python2.7 and linked it:
 +
sudo apt install python2.7 python-numpy
 +
sudo ln -s python2.7 python2
 +
sudo ln -s python2.7 python
 +
 
 +
After this bitbake just complained about missing tools (git, build-essential were already installed):
 +
 
 +
sudo apt install chrpath diffstat gawk makeinfo
  
====Notes====
+
Program test_mcntrl.py that builds x393.h header files from FPGA code that runs during ./setup.py requires numpy and results in error output if missing Still setup.py continues, you need to look through its output few lines after
* Since Poky 1.5.1 switched from u-boot-xlnx to u-boot - some of the u-boot-xlnx reached the upstream
+
* x393
* Since Poky 1.5.1 u-boot introduced Kconfig and currently (as of 2016/01/22) is still migrating from autoconf - keeping both build methods working together.
+
git host: git.elphel.com
* Current u-boot uses the board's device tree blob to make a stripped version for itself - uncompiled device tree files are present in the u-boot sources
+
Clone and checkout: x393
  
==<font color="blue">Compile devicetree</font>==
+
You may install numpy with
<font size='2'>
+
  sudo apt install python3-numpy
  bitbake device-tree
 
</font>
 
====Output====
 
* poky/build/tmp/deploy/images/<machine>/'''devicetree.dtb'''
 
  
==<font color="blue">Build kernel</font>==
+
If you did not notice missing numpy, then during bitbake you will get error "build No rule to make target 'drivers/elphel/x393.o', needed by 'drivers/elphel/built-in.a'.  Stop." in that case you need to install numpy, re-run setup.py (make sure numpy error is gone) then re-create linux kernel links during next build by erasing old ones:
<font size='2'>
+
  bitbake linux-xlnx -c clean
  bitbake linux-xlnx
+
and re-running
or
+
  bitbake linux-xlnx device-tree core-image-elphel393
  bitbake virtual/kernel
 
</font>
 
====Output====
 
* poky/build/tmp/deploy/images/<machine>/'''uImage''' (kernel)
 
  
====Notes====
+
When building php: configure: error: zip support requires ZLIB. Use --with-zlib-dir=<DIR> to specify prefix where ZLIB include and library are located
* Kernel version is 4.0, linux-xlnx
+
sudo apt install zlib1g-dev
* Device Tree Blob is compiled separately in linux-xlnx
 
* Uncompiled device tree files are present in the linux sources provided by linux-xlnx
 
  
==<font color="blue">Build ramdisk.image.gz</font>==
+
==<font color="blue">Compare '''built''' vs '''deployed''' software versions</font>==
 
<font size='2'>
 
<font size='2'>
  bitbake core-image-elphel393
+
  cd elphel393
 +
./check_versions.py root@192.168.0.9
 
</font>
 
</font>
====Output====
 
* poky/build/tmp/deploy/images/<machine>/'''uramdisk.image.gz''' (the file has *.u-boot extension after it's built)
 
 
 
  
==<font color="blue">Notes</font>==
+
{|
* <span style='color:green'>[FIXED]</span> lighttpd-1.4.39, mod-cgi
+
|[[File:Check versions.png|thumb|400px|check_versions.py output in terminal (web-393 revision on PC is ahead of the one installed on the camera)]]
compile warning:
+
|}
implicit declaration of function 'chunkqueue_written'
 
results in lighttpd failing to start if mod-cgi is enabled:
 
Starting Lighttpd Web Server: 2016-01-26 18:42:17: (/data/GIT_fresh/poky/build/tmp/work/cortexa9-vfp-neon-poky-linux-gnueabi/lighttpd/1.4.36-r0/lighttpd-1.4.36/src/plugin.c.169) dlopen() failed for: /usr/lib/mod_cgi.so /usr/lib/mod_cgi.so: undefined symbol: chunkqueue_w
 
2016-01-26 18:42:17: (/data/GIT_fresh/poky/build/tmp/work/cortexa9-vfp-neon-poky-linux-gnueabi/lighttpd/1.4.36-r0/lighttpd-1.4.36/src/server.c.679) loading plugins finally failed
 
lighttpd.
 
Note: why 1.4.36?!!
 
  
<b>solution</b>: exclude the '''0001-mod_cgi-buffers-data-without-bound.patch''' from the file list that comes with the '''poky/meta/recipes-extended/lighttpd_1.4.39.bb'''
+
==<font color="blue">Details for development</font>==
 +
[[Development_for_10393|'''Development for 10393''']]

Latest revision as of 12:22, 1 June 2022

Description

Build the firmware for Elphel 10393 series camera systems from scratch.

Prebuilt releases can be found here.

Use cases

  • Update Linux Kernel when developing a driver
  • Include application to the default firmware package
  • Update Elphel's PHP extension
  • C/C++ applications if developed on PC require a cross-compiler and require Yocto installation or some other alternative.

Other projects written in Python, PHP, Javascript, HTML, C/C++ (gcc is installed on the system board) can be developed on the camera. Installing the framework is not required but it can simplify the development process.

Description

  • Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
  • Yocto Poky revision = 2.7.1 (Warrior) - builds in Kubuntu 20.04. Older OSes are not supported. Roll back to get built in 16.04.
  • Yocto Poky revision = 2.4 (Rocko) - builds in Kubuntu 16.04. Won't build in 18.04 or 20.04.
  • Yocto Poky revision = 2.0 (Jethro)

Features

  • Once everything is built it supports updating software projects (files and binaries) on the target system
    • access is set up in yocto's local.conf (see setup.py output) + ssh-copy-id
    • bitbake some-app -c target_scp from the command line or Eclipse IDE - see examples in elphel-apps-... at Git

Requirements

  • Kubuntu 20.04 x64
  • Kubuntu 18.04 x64 (as of Dec 2018) - Poky: 2.7.1 Warrior (not tested but might build w/o problems)
  • Kubuntu 16.04 x64 (as of Dec 2018) - Poky: 2.4 Rocko, 2.7.1 Warrior

Output files

The names are listed as they appear in the u-boot configuration header file - actual output files have different names:

File(s) Build Command Description
boot.bin bitbake u-boot u-boot as the first stage bootloader = Secondary Program Loader (SPL) that boots u-boot-dtb.img
u-boot-dtb.img bitbake u-boot full size u-boot with a stripped device tree (cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img)
devicetree.dtb bitbake device-tree device tree with listed interfaces, zynq registers, interrupts and drivers
  • elphel393/bootable-images/mmc/devicetree.dtb - mount rootfs from mmc partition
  • elphel393/bootable-images/nand/devicetree.dtb - mount rootfs from NAND flash
uImage bitbake linux-xlnx kernel, drivers
rootfs.ubifs, rootfs.ubi, rootfs.tar.gz bitbake core-image-elphel393 rootfs in different formats - *.ubi* for NAND, *.tar.gz for mmc

The file are located at elphel393/bootable-images/, which is a link to poky's deploy directory elphel393/poky/build/tmp/deploy/images/elphel393/

  • complete set of files for rootfs in NAND Flash: elphel393/bootable-images/nand/
  • complete set of files for rootfs on MMC (micro SD card): elphel393/bootable-images/mmc/

Write files to media and boot

Boot from micro SD card

  • Use recovery or regular μSD card
  • EXT4 partition mounted as /

Detailed instructions

Boot from NAND flash

  • The default boot option - power on.
  • A UBIFS image is written to /dev/mtd4 - is mounted as /

Detailed instructions

Notes

Setup

  • Before building

Installing some dependencies:

sudo apt install git build-essential chrpath diffstat gawk texinfo zlib1g-dev python3-numpy libssl-dev

Installing old python2.7 (bitbake snapshot that we used has problem with python3.8):

sudo apt install python2.7
sudo ln -s python2.7 python2
sudo ln -s python2.7 python
  • Poky 2.7.1 Warrior (Kubuntu 20.04)

git clone -b warrior https://git.elphel.com/Elphel/elphel393.git

If you are a developer and plan to commit changes you should use ssh access instead: git clone -b warrior git@git.elphel.com:Elphel/elphel393.git


cd elphel393
./setup.py
cd poky
. ./oe-init-build-env
bitbake u-boot device-tree linux-xlnx core-image-elphel393

  • Poky 2.4 Rocko (Kubuntu 16.04)

git clone -b rocko https://git.elphel.com/Elphel/elphel393.git
cd elphel393
./setup.py
cd poky
. ./oe-init-build-env
bitbake u-boot device-tree linux-xlnx core-image-elphel393

  • Notes (required actions are already listed above): When trying to run bitbake on a new install got problems with older bitbake version (current build uses specific snapshot) and python3.8. So I install old python2.7 and linked it:
sudo apt install python2.7 python-numpy
sudo ln -s python2.7 python2
sudo ln -s python2.7 python

After this bitbake just complained about missing tools (git, build-essential were already installed):

sudo apt install chrpath diffstat gawk makeinfo

Program test_mcntrl.py that builds x393.h header files from FPGA code that runs during ./setup.py requires numpy and results in error output if missing Still setup.py continues, you need to look through its output few lines after

* x393
git host: git.elphel.com
Clone and checkout: x393

You may install numpy with

sudo apt install python3-numpy

If you did not notice missing numpy, then during bitbake you will get error "build No rule to make target 'drivers/elphel/x393.o', needed by 'drivers/elphel/built-in.a'. Stop." in that case you need to install numpy, re-run setup.py (make sure numpy error is gone) then re-create linux kernel links during next build by erasing old ones:

bitbake linux-xlnx -c clean

and re-running

bitbake linux-xlnx device-tree core-image-elphel393

When building php: configure: error: zip support requires ZLIB. Use --with-zlib-dir=<DIR> to specify prefix where ZLIB include and library are located

sudo apt install zlib1g-dev

Compare built vs deployed software versions

cd elphel393
./check_versions.py root@192.168.0.9

check_versions.py output in terminal (web-393 revision on PC is ahead of the one installed on the camera)

Details for development

Development for 10393