Difference between revisions of "Poky manual"

From ElphelWiki
Jump to: navigation, search
(Required packages)
(Setup)
(38 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
==<font color="blue">Use cases</font>==
 +
* Linux kernel update when adding/developing a driver
 +
* When an application needs to be included into the firmware image archive
 +
* Update Elphel's PHP extension
 +
* C/C++ applications if developed on PC require a cross-compiler and require Yocto installation or some other alternative.
 +
 +
Other projects that use Python, PHP, Javascript, HTML, C/C++ on hardware (compiler is included in the firmware image) do not require this SDK use but it makes the process more convenient, e.g. by copying the built packages straight to the target hardware.
 +
 
==<font color="blue">Description</font>==
 
==<font color="blue">Description</font>==
  
 
* Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
 
* Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
* Yocto Poky revision = 2.0 (Jethro), recently updated to 2.4 (Rocko)
+
* Yocto Poky revision = 2.6 (Thud)
 +
* Yocto Poky revision = 2.4 (Rocko)
 +
* <s>Yocto Poky revision = 2.0 (Jethro)</s>
  
 
==<font color="blue">Features</font>==
 
==<font color="blue">Features</font>==
Line 18: Line 28:
 
** ''bitbake some-app -c target_scp'' from the command line or Eclipse IDE - see examples in '''elphel-apps-...''' at [https://git.elphel.com/Elphel Git]
 
** ''bitbake some-app -c target_scp'' from the command line or Eclipse IDE - see examples in '''elphel-apps-...''' at [https://git.elphel.com/Elphel Git]
  
==<font color="blue">Supported OSes</font>==
+
==<font color="blue">Requirements</font>==
 
+
* Kubuntu 16.04 x64 (as of Dec 2018) for Poky 2.4 Rocko
* Kubuntu 14.04.3 LTS x64 (until June 2016)
+
* Kubuntu 18.04 x64 (as of Dec 2018)
* Kubuntu 16.04 x64 (as of June 2016)
+
* For Poky 2.6 Thud see [http://www.yoctoproject.org/docs/2.6/mega-manual/mega-manual.html Yocto Project Poky 2.6 mega-manual]
  
==<font color="blue">Required packages</font>==
 
* For Poky 2.0 see [http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html Yocto Project Poky 2.0 mega-manual].
 
 
==<font color="blue">Other dependencies</font>==
 
<font size='2'>
 
sudo apt-get install python-numpy
 
</font>
 
 
==<font color="blue">Output files</font>==
 
==<font color="blue">Output files</font>==
 
The names are listed as they appear in the u-boot configuration header file - actual output files have different names:
 
The names are listed as they appear in the u-boot configuration header file - actual output files have different names:
* '''boot.bin''' - u-boot as the first stage bootloader = Secondary Program Loader that boots '''u-boot-dtb.img'''
+
{| class="wikitable"
* '''u-boot-dtb.img''' - full size u-boot with a stripped device tree (''cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img'')
+
|-
* '''devicetree.dtb''' - device tree with listed interfaces, zynq registers, interrupts and drivers.
+
! File(s)
** ''elphel393/bootable-images/mmc/devicetree.dtb'' - mount rootfs from mmc partition
+
! Description
** ''elphel393/bootable-images/nand/devicetree.dtb'' - mount rootfs from NAND flash
+
|-
* '''uImage''' - kernel, drivers
+
| <font color='darkblue'>'''boot.bin'''</font>
* '''rootfs.ubifs''', '''rootfs.ubi''', '''rootfs.tar.gz''' - rootfs in different formats
+
| u-boot as the first stage bootloader = Secondary Program Loader (SPL) that boots '''u-boot-dtb.img'''
 +
|-
 +
| <font color='darkblue'>'''u-boot-dtb.img'''</font>
 +
| full size u-boot with a stripped device tree (''cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img'')
 +
|-
 +
|valign='top'| <font color='darkblue'>'''devicetree.dtb'''</font>
 +
| device tree with listed interfaces, zynq registers, interrupts and drivers
 +
* ''elphel393/bootable-images/mmc/devicetree.dtb'' - mount rootfs from mmc partition
 +
* ''elphel393/bootable-images/nand/devicetree.dtb'' - mount rootfs from NAND flash
 +
|-
 +
| <font color='darkblue'>'''uImage'''</font>
 +
| kernel, drivers
 +
|-
 +
| <font color='darkblue'>'''rootfs.ubifs''', '''rootfs.ubi''', '''rootfs.tar.gz'''</font>
 +
| rootfs in different formats - *.ubi* for NAND, *.tar.gz for mmc
 +
|}
  
They are found at '''elphel393/bootable-images/''', which is a link (for convenience) to poky's deploy directory '''elphel393/poky/build/tmp/deploy/images/elphel393/'''
+
The file are located at '''elphel393/bootable-images/''', which is a link to poky's deploy directory '''elphel393/poky/build/tmp/deploy/images/elphel393/'''
  
 
* complete set of files for rootfs in NAND Flash: ''elphel393/bootable-images/nand/''
 
* complete set of files for rootfs in NAND Flash: ''elphel393/bootable-images/nand/''
Line 60: Line 78:
  
 
==<font color="blue">Setup</font>==
 
==<font color="blue">Setup</font>==
* Poky 2.4 Rocko
+
 
 +
* Poky 2.4 Rocko (Kubuntu 16.04)
 
<font size='2'>
 
<font size='2'>
  git clone -b rocko https://git.elphel.com/Elphel/elphel393.git
+
  git clone -b '''rocko''' https://git.elphel.com/Elphel/elphel393.git
 
  cd elphel393
 
  cd elphel393
 
  ./setup.py
 
  ./setup.py
Line 68: Line 87:
 
  . ./oe-init-build-env
 
  . ./oe-init-build-env
 
  bitbake u-boot device-tree linux-xlnx core-image-elphel393
 
  bitbake u-boot device-tree linux-xlnx core-image-elphel393
</font>
+
<!--''Temporary: linux-xlnx in Rocko depends on branch 'lwir' for x393. Change in projects.json line 42 from 'master' to 'lwir'.''-->
* Poky 2.2 Jethro
 
<font size='2'>
 
git clone https://git.elphel.com/Elphel/elphel393.git # or git clone git@git.elphel.com:Elphel/elphel393.git # if ssh public key is uploaded
 
# then follow the same steps as for Jethro
 
 
</font>
 
</font>
  
'''Notes:'''
+
* Poky 2.6 Thud (Kubuntu 18.04)
* The last line builds all targets.
 
* If python-numpy was not installed
 
 
<font size='2'>
 
<font size='2'>
  x393.h will not be generated > bitbake linux-xnlx will fail.
+
  git clone -b '''thud''' https://git.elphel.com/Elphel/elphel393.git
To recover:
+
cd elphel393
1. sudo apt install python-numpy
+
  ./setup.py
2. run '''bitbake linux-xlnx -c link -f'''
+
  cd poky
</font>
+
. ./oe-init-build-env
* if rerun '''setup.py''' updates all repositories and reruns other procedures.
+
bitbake u-boot device-tree linux-xlnx core-image-elphel393
<font size='2'>
 
  $ ./setup.py
 
  if you notice the kernel got updated then you might need to restart with '''bitbake linux-xlnx -c link -f'''
 
 
</font>
 
</font>
  
Line 106: Line 116:
  
 
===<font color="blue">All targets</font>===
 
===<font color="blue">All targets</font>===
 +
<font size='2'>
 
  bitbake u-boot device-tree linux-xlnx core-image-elphel393
 
  bitbake u-boot device-tree linux-xlnx core-image-elphel393
 
+
</font>
 
===<font color="blue">Bootloader</font>===
 
===<font color="blue">Bootloader</font>===
 
<font size='2'>
 
<font size='2'>
Line 117: Line 128:
  
 
====Notes====
 
====Notes====
* Since Poky 1.5.1: switched from u-boot-xlnx to u-boot - some of the u-boot-xlnx reached the upstream
+
* Current the recipe installs only binaries. We will upgrade u-boot from '''2016.05''' to the current release later.
* Since Poky 1.5.1: u-boot introduced Kconfig and currently (as of 2016/01/22) is still migrating from autoconf - keeping both build methods working together.
+
2016.05 will not compile in Kubuntu 18.04
* Current u-boot uses the board's device tree blob to make a stripped version for itself - uncompiled device tree files are present in the u-boot sources
 
  
 
===<font color="blue">Device tree</font>===
 
===<font color="blue">Device tree</font>===
Line 136: Line 146:
  
 
====Notes====
 
====Notes====
* Kernel version is 4.0, linux-xlnx
+
* linux-xlnx '''rocko''' branch was simply moved to '''thud''' branch
 +
* yocto thud:  Kernel version is 4.9.0, linux-xlnx
 
* Device Tree Blob is compiled separately in linux-xlnx
 
* Device Tree Blob is compiled separately in linux-xlnx
 
* Uncompiled device tree files are present in the linux sources provided by linux-xlnx
 
* Uncompiled device tree files are present in the linux sources provided by linux-xlnx
Line 146: Line 157:
 
====Output====
 
====Output====
 
* '''rootfs.tar.gz'''
 
* '''rootfs.tar.gz'''
* '''rootfs.ubi''','''rootfs.ubifs'''
+
* '''rootfs.ubi''', '''rootfs.ubifs'''
  
 
==<font color="blue">Details for development</font>==
 
==<font color="blue">Details for development</font>==
 
[[Development_for_10393|'''Development for 10393''']]
 
[[Development_for_10393|'''Development for 10393''']]
<br/>
 
<br/>
 
 
==<font color="blue">Notes</font>==
 
* <span style='color:green'>[FIXED]</span> lighttpd-1.4.39, mod-cgi
 
compile warning:
 
implicit declaration of function 'chunkqueue_written'
 
results in lighttpd failing to start if mod-cgi is enabled:
 
Starting Lighttpd Web Server: 2016-01-26 18:42:17: (/data/GIT_fresh/poky/build/tmp/work/cortexa9-vfp-neon-poky-linux-gnueabi/lighttpd/1.4.36-r0/lighttpd-1.4.36/src/plugin.c.169) dlopen() failed for: /usr/lib/mod_cgi.so /usr/lib/mod_cgi.so: undefined symbol: chunkqueue_w
 
2016-01-26 18:42:17: (/data/GIT_fresh/poky/build/tmp/work/cortexa9-vfp-neon-poky-linux-gnueabi/lighttpd/1.4.36-r0/lighttpd-1.4.36/src/server.c.679) loading plugins finally failed
 
lighttpd.
 
Note: why 1.4.36?!!
 
 
<b>solution</b>: exclude the '''0001-mod_cgi-buffers-data-without-bound.patch''' from the file list that comes with the '''poky/meta/recipes-extended/lighttpd_1.4.39.bb'''
 
 
* '''Invalidate chache/checksum on package code changes'''
 
** SRCREV = "${AUTOREV}" - updates the checksum but the cache is not cleared/invalidated?
 
SRCREV <- AUTOREV <- AUTOREV <- "${@bb.fetch2.get_autorev(d)}" (bitbake/lib/bb/__init__.py)
 
 
def get_autorev(d):
 
    #  only not cache src rev in autorev case
 
    if d.getVar('BB_SRCREV_POLICY', True) != "cache":
 
        d.setVar('__BB_DONT_CACHE', '1')
 
    return "AUTOINC"
 
 
BB_DONT_CACHE ?!!!!!!!
 
 
Force to always execute recipe starting from a certain task:
 
recipe-outside-of-task: <font size='2' color='green'><b>do_compile[nostamp]="1"</b></font>
 
or
 
recipe-class-inside-task: <font size='2' color='green'><b>d.setVarFlag('do_compile', 'nostamp', '1')</b></font>
 
 
* clone git repo over https in a recipe
 
SRC_URI = "git://git.elphel.com/project.git;protocol=https;branch=master"
 
 
==<font color="blue">Errors</font>==
 
* '''Aborted, core dumped'''
 
** Got when tried to build gstreamer1.0-plugins-base - error in one of the packages it depends on.
 
** Possible reason - Kubuntu 14.04 - the system libraries are too old? The up-to-date Kubuntu release is 16.04.
 
** Command (example):
 
bitbake libxau-native
 
** log:
 
| x86_64-linux-libtool: link: ar cru .libs/libXau.a  AuDispose.o AuFileName.o AuGetAddr.o AuGetBest.o AuLock.o AuRead.o AuUnlock.o AuWrite.o
 
| *** buffer overflow detected ***: ar terminated
 
| ======= Backtrace: =========
 
| /lib/x86_64-linux-gnu/libc.so.6(+0x7329f)[0x2b732eade29f]
 
| /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x2b732eb75bbc]
 
| /lib/x86_64-linux-gnu/libc.so.6(+0x109a90)[0x2b732eb74a90]
 
| /lib/x86_64-linux-gnu/libc.so.6(+0x108f99)[0x2b732eb73f99]
 
| /lib/x86_64-linux-gnu/libc.so.6(_IO_default_xsputn+0xbc)[0x2b732eae660c]
 
| /lib/x86_64-linux-gnu/libc.so.6(_IO_padn+0xa0)[0x2b732eada600]
 
| /lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x3c4d)[0x2b732eab888d]
 
| /lib/x86_64-linux-gnu/libc.so.6(__vsprintf_chk+0x84)[0x2b732eb74024]
 
| /lib/x86_64-linux-gnu/libc.so.6(__sprintf_chk+0x7d)[0x2b732eb73f7d]
 
| ar[0x406d20]
 
| ar[0x4088df]
 
| ar[0x40bf08]
 
| ar[0x404a86]
 
| ar[0x405003]
 
| ar[0x402a45]
 
| /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x2b732ea8cf45]
 
| ar[0x402dc5]
 
| ======= Memory map: ========
 
| 00400000-00482000 r-xp 00000000 08:01 27273645                          /usr/local/bin/ar
 
| 00682000-00683000 r--p 00082000 08:01 27273645                          /usr/local/bin/ar
 
| 00683000-00684000 rw-p 00083000 08:01 27273645                          /usr/local/bin/ar
 
| 00776000-007e0000 rw-p 00000000 00:00 0                                  [heap]
 
| 2b732e846000-2b732e869000 r-xp 00000000 08:01 26740221                  /lib/x86_64-linux-gnu/ld-2.19.so
 
| 2b732e869000-2b732e86d000 rw-p 00000000 00:00 0
 
| 2b732e898000-2b732e89b000 rw-p 00000000 00:00 0
 
| 2b732ea68000-2b732ea69000 r--p 00022000 08:01 26740221                  /lib/x86_64-linux-gnu/ld-2.19.so
 
| 2b732ea69000-2b732ea6a000 rw-p 00023000 08:01 26740221                  /lib/x86_64-linux-gnu/ld-2.19.so
 
| 2b732ea6a000-2b732ea6b000 rw-p 00000000 00:00 0
 
| 2b732ea6b000-2b732ec25000 r-xp 00000000 08:01 26740235                  /lib/x86_64-linux-gnu/libc-2.19.so
 
| 2b732ec25000-2b732ee25000 ---p 001ba000 08:01 26740235                  /lib/x86_64-linux-gnu/libc-2.19.so
 
| 2b732ee25000-2b732ee29000 r--p 001ba000 08:01 26740235                  /lib/x86_64-linux-gnu/libc-2.19.so
 
| 2b732ee29000-2b732ee2b000 rw-p 001be000 08:01 26740235                  /lib/x86_64-linux-gnu/libc-2.19.so
 
| 2b732ee2b000-2b732ee30000 rw-p 00000000 00:00 0
 
| 2b732ee30000-2b732ee46000 r-xp 00000000 08:01 26741717                  /lib/x86_64-linux-gnu/libgcc_s.so.1
 
| 2b732ee46000-2b732f045000 ---p 00016000 08:01 26741717                  /lib/x86_64-linux-gnu/libgcc_s.so.1
 
| 2b732f045000-2b732f046000 rw-p 00015000 08:01 26741717                  /lib/x86_64-linux-gnu/libgcc_s.so.1
 
| 7fffb978b000-7fffb97ad000 rw-p 00000000 00:00 0                          [stack]
 
| 7fffb97f8000-7fffb97fa000 r-xp 00000000 00:00 0                          [vdso]
 
| ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
 
| ./x86_64-linux-libtool: line 1723:  8035 Aborted                (core dumped) ar cru .libs/libXau.a AuDispose.o AuFileName.o AuGetAddr.o AuGetBest.o AuLock.o AuRead.o AuUnlock.o AuWrite.o
 
| make[2]: *** [libXau.la] Error 134
 
| make[2]: Leaving directory `/GIT/elphel393/poky/build/tmp/work/x86_64-linux/libxau-native/1_1.0.8-r0/build'
 
| make[1]: *** [all-recursive] Error 1
 
| make[1]: Leaving directory `/GIT/elphel393/poky/build/tmp/work/x86_64-linux/libxau-native/1_1.0.8-r0/build'
 
| make: *** [all] Error 2
 
| ERROR: oe_runmake failed
 

Revision as of 16:54, 28 March 2019

Use cases

  • Linux kernel update when adding/developing a driver
  • When an application needs to be included into the firmware image archive
  • Update Elphel's PHP extension
  • C/C++ applications if developed on PC require a cross-compiler and require Yocto installation or some other alternative.

Other projects that use Python, PHP, Javascript, HTML, C/C++ on hardware (compiler is included in the firmware image) do not require this SDK use but it makes the process more convenient, e.g. by copying the built packages straight to the target hardware.

Description

  • Building embedded Linux image for Zynq ARMv7 platform - Elphel 10393 board
  • Yocto Poky revision = 2.6 (Thud)
  • Yocto Poky revision = 2.4 (Rocko)
  • Yocto Poky revision = 2.0 (Jethro)

Features

  • Once everything is built it supports updating software projects (files and binaries) on the target system
    • access is set up in yocto's local.conf (see setup.py output) + ssh-copy-id
    • bitbake some-app -c target_scp from the command line or Eclipse IDE - see examples in elphel-apps-... at Git

Requirements

Output files

The names are listed as they appear in the u-boot configuration header file - actual output files have different names:

File(s) Description
boot.bin u-boot as the first stage bootloader = Secondary Program Loader (SPL) that boots u-boot-dtb.img
u-boot-dtb.img full size u-boot with a stripped device tree (cat u-boot.img some_stripped_devicetree.dtb > u-boot-dtb.img)
devicetree.dtb device tree with listed interfaces, zynq registers, interrupts and drivers
  • elphel393/bootable-images/mmc/devicetree.dtb - mount rootfs from mmc partition
  • elphel393/bootable-images/nand/devicetree.dtb - mount rootfs from NAND flash
uImage kernel, drivers
rootfs.ubifs, rootfs.ubi, rootfs.tar.gz rootfs in different formats - *.ubi* for NAND, *.tar.gz for mmc

The file are located at elphel393/bootable-images/, which is a link to poky's deploy directory elphel393/poky/build/tmp/deploy/images/elphel393/

  • complete set of files for rootfs in NAND Flash: elphel393/bootable-images/nand/
  • complete set of files for rootfs on MMC (micro SD card): elphel393/bootable-images/mmc/

Write files to media and boot

Boot from micro SD card

  • Use recovery or regular μSD card
  • EXT4 partition mounted as /

Detailed instructions

Boot from NAND flash

  • The default boot option - power on.
  • A UBIFS image is written to /dev/mtd4 - is mounted as /

Detailed instructions

Notes

Setup

  • Poky 2.4 Rocko (Kubuntu 16.04)

git clone -b rocko https://git.elphel.com/Elphel/elphel393.git
cd elphel393
./setup.py
cd poky
. ./oe-init-build-env
bitbake u-boot device-tree linux-xlnx core-image-elphel393

  • Poky 2.6 Thud (Kubuntu 18.04)

git clone -b thud https://git.elphel.com/Elphel/elphel393.git
cd elphel393
./setup.py
cd poky
. ./oe-init-build-env
bitbake u-boot device-tree linux-xlnx core-image-elphel393

Compare built vs deployed software versions

cd elphel393
./check_versions.py root@192.168.0.9

Notes:

  • deployed = running on the target system
check_versions.py output in terminal

Build

All targets

bitbake u-boot device-tree linux-xlnx core-image-elphel393

Bootloader

bitbake u-boot

Output

  • boot.bin
  • u-boot-dtb.img

Notes

  • Current the recipe installs only binaries. We will upgrade u-boot from 2016.05 to the current release later.

2016.05 will not compile in Kubuntu 18.04

Device tree

bitbake device-tree

Output

  • devicetree.dtb

Kernel

bitbake linux-xlnx

Output

  • uImage

Notes

  • linux-xlnx rocko branch was simply moved to thud branch
  • yocto thud: Kernel version is 4.9.0, linux-xlnx
  • Device Tree Blob is compiled separately in linux-xlnx
  • Uncompiled device tree files are present in the linux sources provided by linux-xlnx

Rootfs

bitbake core-image-elphel393

Output

  • rootfs.tar.gz
  • rootfs.ubi, rootfs.ubifs

Details for development

Development for 10393