warning: Creating default object from empty value in /home/gwolf/drupal6/modules/taxonomy/ on line 33.

#bananapi → On how compressed files should be used

Submitted by gwolf on Thu, 09/25/2014 - 11:37

I am among the lucky people who got back home from DebConf with a brand new computer: a Banana Pi. Despite the name similarity, it is not affiliated with the very well known Raspberry Pi, although it is a very comparable (although much better) machine: A dual-core ARM A7 system with 1GB RAM, several more on-board connectors, and same form-factor.

I have not yet been able to get it to boot, even from the images distributed on their site (although I cannot complain, I have not devoted more than a hour or so to the process!), but I do have a gripe on how the images are distributed.

I downloaded some images to play with: Bananian, Raspbian, a Scratch distribution, and Lubuntu. I know I have a long way to learn in order to contribute to Debian's ARM port, but if I can learn by doing... ☻

So, what is my gripe? That the three images are downloaded as archive files:

  1. 0 gwolf@mosca『9』~/Download/banana$ ls -hl \
  2. > Lubuntu_For_BananaPi_v3.1.1.tgz Raspbian_For_BananaPi_v3.1.tgz \
  3. > Scratch_For_BananaPi_v1.0.tgz
  4. -rw-r--r-- 1 gwolf gwolf 222M Sep 25 09:52
  5. -rw-r--r-- 1 gwolf gwolf 823M Sep 25 10:02 Lubuntu_For_BananaPi_v3.1.1.tgz
  6. -rw-r--r-- 1 gwolf gwolf 1.3G Sep 25 10:01 Raspbian_For_BananaPi_v3.1.tgz
  7. -rw-r--r-- 1 gwolf gwolf 1.2G Sep 25 10:05 Scratch_For_BananaPi_v1.0.tgz

Now... that is quite an odd way to distribute image files! Specially when looking at their contents:

  1. 0 gwolf@mosca『14』~/Download/banana$ unzip -l
  2. Archive:
  3. Length Date Time Name
  4. --------- ---------- ----- ----
  5. 2032664576 2014-09-17 15:29 bananian-1409.img
  6. --------- -------
  7. 2032664576 1 file
  8. 0 gwolf@mosca『15』~/Download/banana$ for i in Lubuntu_For_BananaPi_v3.1.1.tgz \
  9. > Raspbian_For_BananaPi_v3.1.tgz Scratch_For_BananaPi_v1.0.tgz
  10. > do tar tzvf $i; done
  11. -rw-rw-r-- bananapi/bananapi 3670016000 2014-08-06 03:45 Lubuntu_1404_For_BananaPi_v3_1_1.img
  12. -rwxrwxr-x bananapi/bananapi 3670016000 2014-08-08 04:30 Raspbian_For_BananaPi_v3_1.img
  13. -rw------- bananapi/bananapi 3980394496 2014-05-27 01:54 Scratch_For_BananaPi_v1_0.img

And what is bad about them? That they force me to either have heaps of disk space available (2GB or 4GB for each image) or to spend valuable time extracting before recording the image each time.

Why not just compressing the image file without archiving it? That is,

  1. 0 gwolf@mosca『7』~/Download/banana$ tar xzf Lubuntu_For_BananaPi_v3.1.1.tgz
  2. 0 gwolf@mosca『8』~/Download/banana$ xz Lubuntu_1404_For_BananaPi_v3_1_1.img
  3. 0 gwolf@mosca『9』~/Download/banana$ ls -hl Lubun*
  4. -rw-r--r-- 1 gwolf gwolf 606M Aug 6 03:45 Lubuntu_1404_For_BananaPi_v3_1_1.img.xz
  5. -rw-r--r-- 1 gwolf gwolf 823M Sep 25 10:02 Lubuntu_For_BananaPi_v3.1.1.tgz

Now, wouldn't we need to decompress said files as well? Yes, but thanks to the magic of shell redirections, we can just do it on the fly. That is, instead of having 3×4GB+1×2GB files sitting on my hard drive, I just need to have several files ranging between 145M and I guess ~1GB. Then, it's as easy as doing:

  1. 0 gwolf@mosca『8』~/Download/banana$ dd if=<(xzcat bananian-1409.img.xz) of=/dev/sdd

And the result should be the same: A fresh new card with Bananian ready to fly. Right, right, people using these files need to have xz installed on their systems, but... As it stands now, I can suppose current prospective users of a Banana Pi won't fret about facing a standard Unix tool!

(Yes, I'll forward this rant to the Banana people, it's not just bashing on my blog :-P )

[update] Several people (thanks!) have contacted me stating that I use a bashism: The <(…) construct is specific to Bash. If you want to do this with any other shell, it can be done with a simple pipe:

  1. $ xzcat bananian-1409.img.xz | dd of=/dev/sdd

That allows for less piping to be done on the kernel, and is portable between different shells. Also, a possibility would be:

  1. $ xzcat bananian-1409.img.xz > /dev/sdd

Although that might not be desirable, as it avoids the block-by-block nature of dd. I'm not sure if it makes a realdifference, but it's worth saying :)

And yes, some alternatives for not unarchiving the file — Here in the blog, an anon commenter suggests (respectively, for zip and .tar.gz files):

  1. $ dd if=<(unzip -p of=/dev/sdd
  2. $ dd if=<(tar -xOf Lubuntu_For_BananaPi_v3.1.1.tgz) of=/dev/sdd

And a commenter by IRC suggests:

  1. $ paxtar -xOaf Raspbian_For_BananaPi_v3.1.tgz Raspbian_For_BananaPi_v3_1.img | sudo dd bs=262144 of=/dev/


( categories: )


Submitted by gwolf on Sun, 02/02/2014 - 11:44

Somewhere back in August or September, I pre-ordered a CuBox-i — A nicely finished, completely hackable, and reasonably powerful ARM system, nicely packaged and meant to be used to hack on. A sweet deal!

There are four models (you can see the different models' specs here) — I went for the top one, and bought a CuBox-i4Pro. That means, I have a US$130 nice little box, with 4 ARM7 cores, 2GB RAM, WiFi, and... well, all of its basic goodies and features. For some more details, look at the CuBox-i block diagram.

I got it delivered by early January, and (with no real ARM experience on my side) I finally got to a point where I can, I believe, contribute something to its adoption/usage: How to get a basic Debian system installed and running in it.

The ARM world is quite different to the x86 one: Compatibility is much harder, the computing platform does not self-describe properly, and a kernel must first understand how a specific subarchitecture is before being able to boot on it. Somewhere in the CuBox forums (or was it the IRC channel?) I learnt that the upstream Linux kernel does not yet boot on the i.MX6 chip (although support is rumored to be merged for the 3.14 release), so I am using both a kernel and an uBoot bootloader not built for (or by) Debian people. Besides that, the result I will describe is a kosher Debian install. Yes, I know that my orthodox friends and family will say that 99% kosher is taref... But remember I'm never ever that dogmatic. (yeah, right!)

[update]: Read on if you want to learn the process. If you just want to get the image and start playing with your box, you can go ahead and download it from my space.

Note that there is a prebuilt image you can run if you are so inclined: In the CuBox-i forums and wiki, you will find links to a pre-installed Debian image you can use... But I cannot advise to do so. First, it is IMO quite bloated (you need a 4GB card for a very basic Debian install? Seriously?) Second, it has a whole desktop environment (LXDE, if I recall correctly) and a whole set of packages I will probably not use in this little box. Third, there is a preinstalled user, and that's a no-no (user: debian, password: debian). But, most importantly, fourth: It is a nightly build of the Testing (Jessie) suite... Built back in December. So no, as a Debian Developer, it's not something we should recommend our users to run!

So, in the end and after quite a bit of frustration due to my lack of knowledge, here goes the list of steps I followed:

Using the CuBox
On the i2 and i4 models, you can use it either with a USB keyboard and a HDMI monitor, or by a serial consoles (smaller models do not have a serial console). I don't have a HDMI monitor handy (only a projector), so I prefer to use the serial terminal. Important details to avoid frustration: The USB keyboard has to be connected to the lower USB port, or it will be ignored during the boot process. And make sure your serial terminal is configured not to use hardware flow control. Minicom is configured by default to use hardware flow control, so it was not sending any characters to the CuBox. ^A-O gets you to the Minicom configuration, select Serial port setup, and disable it.
Set up the SD card
I created a 2GB partition, but much less can suffice; I'd leave it at least to 1GB to do the base install, although it can be less once the system is set up (more on this later). Partition and format using your usual tools (fdisk+mke2fs, or gparted, or whatever suits your style).
Install the bootloader
I followed up the instructions on this CuBox-i forums thread to get the SPL and uBoot bootloader running. In short, from this Google Drive folder, download the SPL-U-Boot.img.xz file, uncompress it (xz --decompress SPL-U-Boot.img.xz), and write it to the SD card just after the partition map: As root,
# dd if=SPL-U-Boot.img of=/dev/mmcblk0 bs=1024 seek=1 skip=1.
Actually, to be honest: As I wanted something basic to be able to debug from, I downloaded (from the same Google Drive) the busybox.img.gz file. That's a bit easier to install from: xz --decompress busybox.img.xz, and just dump it into the SD from the beginning (as it does already include a partition table):
# dd if=busybox.img of=/dev/mmcblk0
This card is already bootable and minimal, and allows to debug some bits from the CuBox-i itself (as we will see shortly).
After this step, I created a second partition, as I said earlier. So, my mmcblk0p1 partition holds Busybox, and the second will hold Debian. We are still working from the x86 system, so we mount the SD card in /media/mmcblk0p2
Installing the base system
Without debian-installer to do the heavy lifting, I went for debootstrap. As I ran it from my PC, debootstrap's role will be for this first stage only to download and do a very initial pre-unpacking of the files: Bootstrapping a foreign architecture implies, right, using the --foreign switch:
debootstrap --foreign --arch=armhf wheezy /media/mmcblk0p2
You can add some packages you often use by specifying --include=foo,bar,baz
So, take note notes: This board is capable of running the armhf architecture (HF for Hardware Float). It can also run armel, but I understand it is way slower.
First boot (with busybox)
So, once debootstrap finishes, you are good to go to the real hardware! Unmount the SD card, put it in the little guy, plug your favorite console in (I'm using the serial port), and plug the power in! You should immediately see something like:
  1. U-Boot SPL 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
  2. Boot Device: SD1
  3. reading u-boot.img
  4. Load image from RAW...
  7. U-Boot 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
  9. CPU: Freescale i.MX6Q rev1.2 at 792 MHz
  10. Reset cause: POR
  11. Board: MX6-CuBox-i
  12. DRAM: 2 GiB
  13. MMC: FSL_SDHC: 0
  14. In: serial
  15. Out: vga
  16. Err: vga
  17. Net: phydev = 0x0
  18. Phy not found
  19. PHY reset timed out
  20. FEC
  21. (Re)start USB...
  22. USB0: USB EHCI 1.00
  23. scanning bus 0 for devices... 1 USB Device(s) found
  24. scanning usb for storage devices... 0 Storage Device(s) found
  25. scanning usb for ethernet devices... 0 Ethernet Device(s) found
  26. Hit any key to stop autoboot: 3

Let it boot (that means, don't stop autoboot), and you will soon see a familiar #, showing you are root in the busybox environment. Great! Now, mount the Debian partition:
# mount /dev/mmcblk0p2 /mnt
Finishing debootstrap's task
With everything in place, it's time for debootstrap to work. Chroot into the Debian partition:
# chroot /mnt
And ask Debootstrap to finish what it started:
# /debootstrap/debootstrap --second-stage
Be patient, as this step takes quite a bit to be finished.
Some extra touches...
After this is done, your Debian system is almost ready to be booted into. Why almost? Because it still does not have any users, does not know its own name nor knows I want to use it via a serial terminal, and does not know how the filesystems should be mounted and made available. And having a Debian system means having its very extensive software repository collection handy! Five very simple tasks to fix:
  1. Set a password for root:
    1. # passwd
    2. Enter new UNIX password:
    3. Retype new UNIX password:
    4. passwd: password updated successfully
  2. Setting your hostname is trivial:
    1. # echo > /etc/hostname

    So you have now a usable root user, and when you boot with it you can create further users.
  3. Now, to get the serial console working (you might not need it, if you use the CuBox-i via keyboard+monitor) add a line to /etc/inittab specifying the details of the serial console. You can just do this:
    1. # echo 'T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt100' >> /etc/inittab
  4. Create a /etc/fstab specifying how the system will be laid out. Right now, it is quite trivial (and in fact, I used my machine for some time without even thinking about this, just using the parameter provided to the kernel, this setting will just give you an easier and even faster experience):
    1. # cat > /etc/fstab
    2. /dev/mmcblk0p2 / ext3 noatime 0 0
    3. /dev/mmcblk0p1 /boot ext2 ro 0 0
    4. proc /proc proc defaults 0 0
    5. tmpfs /tmp tmpfs defaults 0 0
    6. tmpfs /run tmpfs defaults 0 0
  5. Tell your computer where to get the Debian packages. I suggest you use the meta-mirror, which will resolve to the mirror closest to you, but you can of course choose from the worldwide list of Debian mirrors.
    # echo deb wheezy main > /etc/apt/sources.list
    # echo deb-src wheezy main > /etc/apt/sources.list
Boot into Debian!
So, ready to boot Debian? Ok, first exit the chroot shell, to go back to the Busybox shell, unmount the Debian partition, and set the root partition read-only:
  1. # exit
  2. # umount /mnt
  3. # mount / -o remount,ro

Disconnect and connect power, and now, do interrupt the boot process when you see the Hit any key to stop automount prompt. To see the configuration of uboot, you can type printenv — We will only modify the parameters given to the kernel:
  1. CuBox-i U-Boot > setenv root /dev/mmcblk0p2 rootfstype=ext3 ro rootwait
  2. CuBox-i U-Boot > boot

So, the kernel will load, and a minimal Debian system will be initialized. In my case, I get the following output:
  1. ** File not found /boot/busyEnv.txt **
  2. 4703740 bytes read in 390 ms (11.5 MiB/s)
  3. ## Booting kernel from Legacy Image at 10000000 ...
  4. Image Name: Linux-3.0.35-8
  5. Image Type: ARM Linux Kernel Image (uncompressed)
  6. Data Size: 4703676 Bytes = 4.5 MiB
  7. Load Address: 10008000
  8. Entry Point: 10008000
  9. Verifying Checksum ... OK
  10. Loading Kernel Image ... OK
  12. Starting kernel ...
  14. Unable to get enet.0 clock
  15. pwm-backlight pwm-backlight.0: unable to request PWM for backlight
  16. pwm-backlight pwm-backlight.1: unable to request PWM for backlight
  17. _regulator_get: get() with no identifier
  18. mxc_sdc_fb mxc_sdc_fb.2: NO mxc display driver found!
  19. INIT: version 2.88 booting
  20. [info] Using makefile-style concurrent boot in runlevel S.
  21. [....] Starting the hotplug events dispatcher: udevd. ok
  22. [....] Synthesizing the initial hotplug events...done.
  23. [....] Waiting for /dev to be fully populated...done.
  24. [....] Activating swap...done.
  25. [....] Cleaning up temporary files... /tmp. ok
  26. [....] Activating lvm and md swap...done.
  27. [....] Checking file systems...fsck from util-linux 2.20.1
  28. done.
  29. [....] Mounting local filesystems...done.
  30. [....] Activating swapfile swap...done.
  31. [....] Cleaning up temporary files.... ok
  32. [....] Setting kernel variables ...done.
  33. [....] Configuring network interfaces...done.
  34. [....] Cleaning up temporary files.... ok
  35. [....] Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix. ok
  36. INIT: Entering runlevel: 2
  37. [info] Using makefile-style concurrent boot in runlevel 2.
  38. [....] Starting enhanced syslogd: rsyslogd. ok
  39. [....] Starting periodic command scheduler: cron. ok
  41. Debian GNU/Linux 7 ttymxc0
  43. cubox-i login:

And that's it, the system is live and ready for my commands!

So, how big is this minimal Debian installed system? I cheated a bit on this, as I had already added emacs and screen to the system, so yours will be a small bit smaller. But anyway — Lets clear our cache of downloaded packages, and see the disk usage information:

  1. root@cubox-i:~# apt-get clean
  2. root@cubox-i:~# df -h
  3. Filesystem Size Used Avail Use% Mounted on
  4. rootfs 689M 228M 427M 35% /
  5. /dev/root 689M 228M 427M 35% /
  6. devtmpfs 881M 0 881M 0% /dev
  7. tmpfs 177M 144K 177M 1% /run
  8. tmpfs 5.0M 0 5.0M 0% /run/lock
  9. tmpfs 353M 0 353M 0% /run/shm
  10. tmpfs 881M 0 881M 0% /tmp

So, instead of a 4GB install, we have a 228MB one. Great improvement!

For this first boot, and until you set up a way to automatically (or configure it to be static) determine the network configuration, you can use dhclient eth0 to request an IP address via the wired network port (configuring the wireless network is a bit more involved; I suggest you install the wicd-curses package to help on that regard). With the network working, update the Debian package lists:

# apt-get update
Get:1 wheezy Release.gpg [1672 B]
Get:2 wheezy Release [168 kB]
Get:3 wheezy/main Sources [5956 kB]
Get:4 wheezy/main armhf Packages [5691 kB]              
Get:5 wheezy/main Translation-en [3849 kB]              
Fetched 15.7 MB in 1min 27s (180 kB/s)                                         
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Yay, all of Debian is now at your fingertips! Now, lets get it to do something useful, in a most Debianic way!

[note]: I have tried to keep this as true as possible to the real install. I have modified this text every now and then, looking at ways to make it a little bit better. So, excuse me if you find any inconsistencies in the instructions! :)

[update]: I finally followed through the instructions again and produced a downloadable image, where I did all of this work, and you can just download it and play with your CuBox-i! You can download it from my space. You will find there instructions on how to get it installed.

( categories: )
Syndicate content