summaryrefslogtreecommitdiff
path: root/util/cbfstool/cbfs_image.c
AgeCommit message (Collapse)Author
2018-05-04cbfs: Rename CBFS_TYPE_PAYLOAD to CBFS_TYPE_SELFPatrick Rudolph
In preparation of having FIT payloads, which aren't converted to simple ELF, rename the CBFS type payload to actually show the format the payload is encoded in. Another type CBFS_TYPE_FIT will be added to have two different payload formats. For now this is only a cosmetic change. Change-Id: I39ee590d063b3e90f6153fe655aa50e58d45e8b0 Signed-off-by: Patrick Rudolph <patrick.rudolph@9elements.com> Reviewed-on: https://review.coreboot.org/25986 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Reviewed-by: Aaron Durbin <adurbin@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2017-10-28cbfstool: Print compression algorithmDaisuke Nojiri
This patch adds a column to the print command to show the compression algorithm used for the file. Name Offset Type Size Comp fallback/romstage 0x0 stage 56236 none ecrw 0xf2380 raw 62162 LZMA (131072 decompressed) BUG=b:66956286 BRANCH=none TEST=Run 'cbfstool image.bin print' Change-Id: I4bbb60ab467adac4ae5486ddafec86ad9682a40e Signed-off-by: Daisuke Nojiri <dnojiri@chromium.org> Reviewed-on: https://review.coreboot.org/22196 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2017-10-10util/cbfstool: Add "truncate" commandPatrick Georgi
It does the opposite to "expand", removing a trailing empty file from CBFS. It also returns the size of the CBFS post processing on stdout. BUG=b:65853903 BRANCH=none TEST=`cbfstool test.bin truncate -r FW_MAIN_A` removes the trailing empty file in FW_MAIN_A. Without a trailing empty file, the region is left alone (tested using COREBOOT which comes with a master header pointer). Change-Id: I0c747090813898539f3428936afa9d8459adee9c Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/21608 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2017-09-20util/cbfstool: Add "expand" command to make CBFS span an fmap regionPatrick Georgi
vboot images come with multiple regions carrying CBFS file systems. To expedite hashing (from slow flash memory), the FW_MAIN_* regions are truncated since they typically have pretty large unused space at the end that is of no interest. For test purposes it can be useful to re-engage that space, so add a command that creates a new empty file entry covering that area (except for the last 4 bytes for the master header pointer, as usual). BUG=b:65853903 BRANCH=none TEST=`cbfstool test.bin expand -r FW_MAIN_A` creates a new empty file of the expected size on a Chrome OS firmware image. Change-Id: I160c8529ce4bfcc28685166b6d9035ade4f6f1d1 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/21598 Reviewed-by: Aaron Durbin <adurbin@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2017-03-20util/cbfstool: avoid memleaks and off-by-onesPatrick Georgi
Change-Id: Iac136a5dfe76f21aa7c0d5ee4e974e50b955403b Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Found-by: scan-build 3.8 Reviewed-on: https://review.coreboot.org/18134 Tested-by: build bot (Jenkins) Reviewed-by: Nico Huber <nico.h@gmx.de>
2017-01-12util/cbfstool: Add cbfs-compression-toolPatrick Georgi
cbfs-compression-tool provides a way to benchmark the compression algorithms as used by cbfstool (and coreboot) and allows to pre-compress data for later consumption by cbfstool (once it supports the format). For an impression, the benchmark's results on my machine: measuring 'none' compressing 10485760 bytes to 10485760 took 0 seconds measuring 'LZMA' compressing 10485760 bytes to 1736 took 2 seconds measuring 'LZ4' compressing 10485760 bytes to 41880 took 0 seconds And a possible use for external compression, parallel and non-parallel (60MB in 53 files compressed to 650KB on a machine with 40 threads): $ time (ls -1 *.* |xargs -n 1 -P $(nproc) -I '{}' cbfs-compression-tool compress '{}' out/'{}' LZMA) real 0m0.786s user 0m11.440s sys 0m0.044s $ time (ls -1 *.* |xargs -n 1 -P 1 -I '{}' cbfs-compression-tool compress '{}' out/'{}' LZMA) real 0m10.444s user 0m10.280s sys 0m0.064s Change-Id: I40be087e85d09a895b1ed277270350ab65a4d6d4 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: https://review.coreboot.org/18099 Tested-by: build bot (Jenkins) Reviewed-by: Martin Roth <martinroth@google.com>
2016-12-15util/cbfstool: Fix memory leakPatrick Georgi
Change-Id: I66cb1c88155ef58610bacfb899e0132e4143c7ac Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Found-by: Coverity Scan #1325836 Reviewed-on: https://review.coreboot.org/17859 Reviewed-by: Nico Huber <nico.h@gmx.de> Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2016-12-15util/cbfstool: Add NULL-ptr checkPatrick Georgi
Change-Id: I8b5caf5423135fe683a24db6700b895a2685cb98 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Found-by: Coverity Scan #1323507 Reviewed-on: https://review.coreboot.org/17858 Reviewed-by: Nico Huber <nico.h@gmx.de> Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2016-12-04cbfstool: Fix off-by-one error in checking hash_typeFurquan Shaikh
Change-Id: Iaf208705d0cd450288af721d53053b2d3407a336 Found-by: Coverity Scan #1325836 Signed-off-by: Furquan Shaikh <furquan@chromium.org> Reviewed-on: https://review.coreboot.org/17698 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-08-10util/cbfstool: Initialize elf_writer pointer to avoid crashJonathan Neuschäfer
If some error happens in cbfs_payload_make_elf, the code jumps to "out", and elf_writer_destroy(ew) is called. This may happen before an elf writer is allocated. To avoid accessing an uninitialized pointer, initialize ew to NULL; elf_writer_destroy will perform no action in this case. Change-Id: I5f1f9c4d37f2bdeaaeeca7a15720c7b4c963d953 Reported-By: Coverity Scan (1361475) Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Reviewed-on: https://review.coreboot.org/16124 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-08-08cbfstool/cbfs_image: Fix resource leak for tbuffFurquan Shaikh
Change-Id: I1f4626e1bda92af38e7967d7e05a4c7143942cf6 Signed-off-by: Furquan Shaikh <furquan@google.com> Reviewed-on: https://review.coreboot.org/16074 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-08-08cbfstool/cbfs_image: Check for return value of buffer_createFurquan Shaikh
Free any buffers if required. Change-Id: Iccd435dba51275d875a5fdb5649cdcd0541fd84c Signed-off-by: Furquan Shaikh <furquan@google.com> Found-by: Coverity Scan # 1361254 Reviewed-on: https://review.coreboot.org/16073 Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-08-08cbfstool/cbfs_image: Fix resource leak for segsFurquan Shaikh
Free segs whenever returning from cbfs_payload_make_elf() Change-Id: I0dd722dd488723cecffe1f5621244bb0344056a6 Signed-off-by: Furquan Shaikh <furquan@google.com> Found-by: Coverity Scan # 1361268 Reviewed-on: https://review.coreboot.org/16072 Tested-by: build bot (Jenkins) Reviewed-by: Antonello Dettori <dev@dettori.io> Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-08-08cbfstool/cbfs_image: Initialze empty_sz to 0Furquan Shaikh
Change-Id: I8b9cfe56b5893ba11047fcc1a6727e7e12a15772 Signed-off-by: Furquan Shaikh <furquan@google.com> Found-by: Coverity Scan # 1361276 Reviewed-on: https://review.coreboot.org/16071 Reviewed-by: Aaron Durbin <adurbin@chromium.org> Tested-by: build bot (Jenkins)
2016-07-02cbfstool: Require "-m ARCH" to extract payloads and stagesAntonello Dettori
Require the user to specify which architecture the payload/stage was built for before extracting it. Change-Id: I8ffe90a6af24e76739fd25456383a566edb0da7e Signed-off-by: Antonello Dettori <dev@dettori.io> Reviewed-on: https://review.coreboot.org/15438 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-06-17cbfstool: Extract payload in ELFAntonello Dettori
Implement function that automatically converts a SELF payload, extracted from the CBFS, into an ELF file. The code has been tested on the following payloads: Working: GRUB, FILO, SeaBIOS, nvramcui, coreinfo and tint Currently not working: none Change-Id: I51599e65419bfa4ada8fe24b119acb20c9936227 Signed-off-by: Antonello Dettori <dettori.an@gmail.com> Reviewed-on: https://review.coreboot.org/15139 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-05-28util/cbfstool: Include commonlib/helpers.h in common.hFurquan Shaikh
This avoids re-declaring common macros like ARRAY_SIZE, MIN, MAX and ALIGN. Also removes the issues around including both files in any tool. Also, fix comparison error in various files by replacing int with size_t. Change-Id: I06c763e5dd1bec97e8335499468bbdb016eb28e5 Signed-off-by: Furquan Shaikh <furquan@google.com> Reviewed-on: https://review.coreboot.org/14978 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-02-22cbfs: Add LZ4 in-place decompression support for pre-RAM stagesJulius Werner
This patch ports the LZ4 decompression code that debuted in libpayload last year to coreboot for use in CBFS stages (upgrading the base algorithm to LZ4's dev branch to access the new in-place decompression checks). This is especially useful for pre-RAM stages in constrained SRAM-based systems, which previously could not be compressed due to the size requirements of the LZMA scratchpad and bounce buffer. The LZ4 algorithm offers a very lean decompressor function and in-place decompression support to achieve roughly the same boot speed gains (trading compression ratio for decompression time) with nearly no memory overhead. For now we only activate it for the stages that had previously not been compressed at all on non-XIP (read: non-x86) boards. In the future we may also consider replacing LZMA completely for certain boards, since which algorithm wins out on boot speed depends on board-specific parameters (architecture, processor speed, SPI transfer rate, etc.). BRANCH=None BUG=None TEST=Built and booted Oak, Jerry, Nyan and Falco. Measured boot time on Oak to be about ~20ms faster (cutting load times for affected stages almost in half). Change-Id: Iec256c0e6d585d1b69985461939884a54e3ab900 Signed-off-by: Julius Werner <jwerner@chromium.org> Reviewed-on: https://review.coreboot.org/13638 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-02-12util/cbfstool: Improve heuristic for cbfs header pointer protectionPatrick Georgi
cbfstool has a routine to deal with old images that may encourage it to overwrite the master header. That routine is triggered for "cbfstool add-master-header" prepared images even though these are not at risk, and - worse - destroys the chain structure (through a negative file length), so avoid touching such images. Change-Id: I9d0bbe3e6300b9b9f3e50347737d1850f83ddad8 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: https://review.coreboot.org/13672 Tested-by: build bot (Jenkins) Reviewed-by: Martin Roth <martinroth@google.com>
2016-01-28util/cbfstool: add 'compact' commandAaron Durbin
While assembling CBFS images within the RW slots on Chrome OS machines the current approach is to 'cbfstool copy' from the RO CBFS to each RW CBFS. Additional fixups are required such as removing unneeded files from the RW CBFS (e.g. verstage) as well as removing and adding back files with the proper arguments (FSP relocation as well as romstage XIP relocation). This ends up leaving holes in the RW CBFS. To speed up RW CBFS slot hashing it's beneficial to pack all non-empty files together at the beginning of the CBFS. Therefore, provide the 'compact' command which bubbles all the empty entries to the end of the CBFS. Change-Id: I8311172d71a2ccfccab384f8286cf9f21a17dec9 Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: https://review.coreboot.org/13479 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2016-01-28util/cbfstool: add machine parseable printAaron Durbin
In order to more easily process the output of 'cbfstool print' with other tools provide a -k option which spits out the tab-separated header and fields: Name Offset Type Metadata Size Data Size Total Size ALIGN_UP(Offset + Total Size, 64) would be the start of the next entry. Also, one can analzye the overhead and offsets of each file more easily. Example output (note: tabs aren't in here): $ ./coreboot-builds/sharedutils/cbfstool/cbfstool test.serial.bin print -r FW_MAIN_A -k Performing operation on 'FW_MAIN_A' region... Name Offset Type Metadata Size Data Size Total Size cmos_layout.bin 0x0 cmos_layout 0x38 0x48c 0x4c4 dmic-2ch-48khz-16b.bin 0x500 raw 0x48 0xb68 0xbb0 dmic-2ch-48khz-32b.bin 0x10c0 raw 0x48 0xb68 0xbb0 nau88l25-2ch-48khz-24b.bin 0x1c80 raw 0x48 0x54 0x9c ssm4567-render-2ch-48khz-24b.bin 0x1d40 raw 0x58 0x54 0xac ssm4567-capture-4ch-48khz-32b.bin 0x1e00 raw 0x58 0x54 0xac vbt.bin 0x1ec0 optionrom 0x38 0x1000 0x1038 spd.bin 0x2f00 spd 0x38 0x600 0x638 config 0x3540 raw 0x38 0x1ab7 0x1aef revision 0x5040 raw 0x38 0x25e 0x296 font.bin 0x5300 raw 0x38 0x77f 0x7b7 vbgfx.bin 0x5ac0 raw 0x38 0x32f8 0x3330 locales 0x8e00 raw 0x28 0x2 0x2a locale_en.bin 0x8e40 raw 0x38 0x29f6 0x2a2e u-boot.dtb 0xb880 mrc_cache 0x38 0xff1 0x1029 (empty) 0xc8c0 null 0x64 0xadf4 0xae58 fallback/ramstage 0x17740 stage 0x38 0x15238 0x15270 (empty) 0x2c9c0 null 0x64 0xd2c4 0xd328 fallback/payload 0x39d00 payload 0x38 0x12245 0x1227d cpu_microcode_blob.bin 0x4bf80 microcode 0x60 0x17000 0x17060 (empty) 0x63000 null 0x28 0x37cf98 0x37cfc0 Change-Id: I1c5f8c1b5f2f980033d6c954c9840299c6268431 Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: https://review.coreboot.org/13475 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2016-01-27cbfstool: Fix broken alignment because of flashmapWerner Zeh
With the introduction of flashmap cbfs alignment of files gets broken because flashmap is located at the beginning of the flash and cbfstool didn't take care about that offset. This commit fixes the alignment in cbfs. Change-Id: Idebb86d4c691b49a351a402ef79c62d31622c773 Signed-off-by: Werner Zeh <werner.zeh@siemens.com> Reviewed-on: https://review.coreboot.org/13417 Reviewed-by: Aaron Durbin <adurbin@chromium.org> Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2016-01-06cbfstool: keep cbfs master header pointerPatrick Georgi
Adding new files overwrote the header with the empty file (ie 0xff), so carve out some space. BUG=chromium:445938 BRANCH=none TEST=none Change-Id: I91c292df381c2bac41c6cb9dda74dae99defd81d Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/12789 Tested-by: build bot (Jenkins)
2016-01-06cbfstool: Adapt "cbfstool copy" to only use fmap regions.Patrick Georgi
These need to go together, so the commit became a bit larger than typial. - Add an option -R for the copy source fmap region. Use: cbfstool copy -r target-region -R source-region. - Don't generate a CBFS master header because for fmap regions, we assume that the region starts with a file header. Use cbfstool add-master-header to add it afterwards, if necessary. - Don't copy files of type "cbfs master header" (which are what cbfstool add-master-header creates) - Leave room for the master header pointer - Remove -D command line option as it's no longer used. BUG=chromium:445938 BRANCH=none TEST=Manual test on image and integration test w/ bundle_firmware changes. CQ-DEPEND=CL:313770,CL:313771 Change-Id: I2a11cda42caee96aa763f162b5f3bc11bb7992f9 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: https://review.coreboot.org/12788 Tested-by: build bot (Jenkins)
2016-01-06cbfstool: Use buffer over offset/size pair for cbfs_copy_instancePatrick Georgi
This allows adding support for FMAP based cbfstool copy more easily. BUG=chromium:445938 Change-Id: I72e7bc4da7d27853e324400f76f86136e3d8726e Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/12787 Tested-by: build bot (Jenkins)
2015-11-20cbfstool: Fix build error with clang when comparing enumzbao
If HOSTCC=clang, the -Wtautological-constant-out-of-range-compare is set automaticaaly. That assume the value of type enum is in the defined range. Then testing if a type enum is out of range causes build error. Error: coreboot/util/cbfstool/cbfs_image.c:1387:16: error: comparison of constant 4 with expression of type 'enum vb2_hash_algorithm' is always false [-Werror,-Wtautological-constant-out-of-range-compare] if (hash_type >= CBFS_NUM_SUPPORTED_HASHES) ~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. clang version: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Target: x86_64-unknown-freebsd10.2 Thread model: posix Change-Id: I3e1722bf6f9553793a9f0c7f4e790706b6938522 Signed-off-by: zbao <fishbaozi@gmail.com> Reviewed-on: http://review.coreboot.org/12330 Tested-by: build bot (Jenkins) Reviewed-by: Martin Roth <martinroth@google.com>
2015-11-13cbfstool: Allows mixed-state fmap regions to workPatrick Georgi
When using FMAP regions (with option -r) that were generated with a master header (as done by cbfstool copy, eg. in Chrome OS' build system), there were differences in interpretation of the master header's fields. Normalize for that by not sanity-checking the master header's size field (there are enough other tests) and by dealing with region offsets properly. BUG=chromium:445938 BRANCH=tot TEST=`cbfstool /build/veyron_minnie/firmware/image.dev.bin print -r FW_MAIN_A` shows that region's directory (instead of claiming that there's no CBFS at all, or showing an empty directory). Change-Id: Ia840c823739d4ca144a7f861573d6d1b4113d799 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: 0e5364d291f45e4705e83c0331e128e35ab226d3 Original-Change-Id: Ie28edbf55ec56b7c78160000290ef3c57fda0f0e Original-Signed-off-by: Patrick Georgi <pgeorgi@google.com> Original-Reviewed-on: https://chromium-review.googlesource.com/312210 Original-Commit-Ready: Patrick Georgi <pgeorgi@chromium.org> Original-Tested-by: Patrick Georgi <pgeorgi@chromium.org> Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/12416 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-10-31tree: drop last paragraph of GPL copyright headerPatrick Georgi
It encourages users from writing to the FSF without giving an address. Linux also prefers to drop that and their checkpatch.pl (that we imported) looks out for that. This is the result of util/scripts/no-fsf-addresses.sh with no further editing. Change-Id: Ie96faea295fe001911d77dbc51e9a6789558fbd6 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: http://review.coreboot.org/11888 Tested-by: build bot (Jenkins) Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2015-10-29cbfstool: extract rmodules as ELFs properlyAaron Durbin
With the previous ELF stage extract support the resulting ELF files wouldn't handle rmodules correctly in that the rmodule header as well as the relocations were a part of the program proper. Instead, try an initial pass at converting the stage as if it was an rmodule first. If it doesn't work fall back on the normal ELF extraction. TEST=Pulled an rmodule out of Chrome OS shellball. Manually matched up the metadata and relocations. Change-Id: Iaf222f92d145116ca4dfaa955fb7278e583161f2 Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/12222 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com> Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2015-10-29cbfstool: create ELF files when extracting stagesAaron Durbin
Instead of dumping the raw stage data when cbfstool extract is used on stage create an equivalent ELF file. Because there isn't a lot of information within a stage file only a rudimentary ELF can be created. Note: this will break Chrome OS' current usage of extract since the file is no longer a cbfs_stage. It's an ELF file. TEST=Extracted romstage from rom. Change-Id: I8d24a7fa4c5717e4bbba5963139d0d9af4ef8f52 Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/12219 Reviewed-by: Patrick Georgi <pgeorgi@google.com> Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2015-10-29cbfstool: add optional -m ARCH to extractAaron Durbin
In order to prepare allowing for one to extract a stage into an ELF file provide an optional -m ARCH option. This allows one to indicate to cbfstool what architecture type the ELF file should be in. Longer term each stage and payload will have an attribute associated with it which indicates the attributes of the executable. Change-Id: Id190c9719908afa85d5a3b2404ff818009eabb4c Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/12217 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com> Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2015-10-27cbfstool: decompress stage files on extractionAaron Durbin
In order to actually do something useful with the resulting file after being extracted decompress stage files' content. That way one can interrogate the resulting file w/o having to decompress on the fly. Note: This change will cause an unexpected change to Chrome OS devices which package up individual stage files in the RW slots w/o using cbfs. The result will be that compressed stages are now decompressed. Longer term is to turn these files into proper ELF files on the way out. Change-Id: I373ecc7b924ea21af8d891a8cb8f01fd64467360 Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/12174 Reviewed-by: Patrick Georgi <pgeorgi@google.com> Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2015-10-25cbfstool: have decompress functions provide ouput data sizeAaron Durbin
Currently cbfs stage files that are compressed do not have the decompressed size readily available. Therefore there's no good way to know actual size of data after it is decompressed. Optionally return the decompressed data size if requested. Change-Id: If371753d28d0ff512118d8bc06fdd48f4a0aeae7 Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/12173 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-10-17cbfstool: Fix removing and adding file with same nameAaron Durbin
Currently, cbfstool regressed that removing a file from CBFS the space is marked as empty but the filename is still shown, preventing adding a file with the same name again. [1] ``` $ echo a > a $ echo b > b $ ./util/cbfstool/cbfstool test.rom create -m x86 -s 1024 Created CBFS (capacity = 920 bytes) $ ./util/cbfstool/cbfstool test.rom add -f a -n a -t raw $ ./util/cbfstool/cbfstool test.rom add -f b -n b -t raw $ cp test.rom test.rom.original $ ./util/cbfstool/cbfstool test.rom remove -n $ diff -up <(hexdump -C test.rom.original) <(hexdump -C test.rom) --- /dev/fd/63 2015-08-07 08:43:42.118430961 -0500 +++ /dev/fd/62 2015-08-07 08:43:42.114430961 -0500 @@ -1,4 +1,4 @@ -00000000 4c 41 52 43 48 49 56 45 00 00 00 02 00 00 00 50 |LARCHIVE.......P| +00000000 4c 41 52 43 48 49 56 45 00 00 00 02 ff ff ff ff |LARCHIVE........| 00000010 00 00 00 00 00 00 00 28 61 00 00 00 00 00 00 00 |.......(a.......| 00000020 00 00 00 00 00 00 00 00 61 0a ff ff ff ff ff ff |........a.......| 00000030 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................| $ ./util/cbfstool/cbfstool test.rom add -f c -n c -t raw $ ./util/cbfstool/cbfstool test.rom print test.rom: 1 kB, bootblocksize 0, romsize 1024, offset 0x0 alignment: 64 bytes, architecture: x86 Name Offset Type Size c 0x0 raw 2 b 0x40 raw 2 (empty) 0x80 null 792 ``` So it is “deteled” as the type changed. But the name was not changed to match the *(empty)* heuristic. So also adapt the name when removing a file by writing a null byte to the beginning of the name, so that the heuristic works. (Though remove doesn't really clear contents.) ``` $ ./util/cbfstool/cbfstool test.rom remove -n c $ ./util/cbfstool/cbfstool test.rom print test.rom: 1 kB, bootblocksize 0, romsize 1024, offset 0x0 alignment: 64 bytes, architecture: x86 Name Offset Type Size (empty) 0x0 null 2 b 0x40 raw 2 (empty) 0x80 null 792 ``` [1] http://www.coreboot.org/pipermail/coreboot/2015-August/080201.html Change-Id: I033456ab10e3e1b402ac2374f3a887cefd3e5abf Signed-off-by: Aaron Durbin <adurbin@chromium.org> Signed-off-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-on: http://review.coreboot.org/11632 Tested-by: build bot (Jenkins) Tested-by: Raptor Engineering Automated Test Stand <noreply@raptorengineeringinc.com>
2015-10-17cbfstool/cbfs_image.c: Just use one space before `=`Paul Menzel
Change-Id: Id31c889d1e83e7ddfb0f0f98b78601f37b71cfa2 Signed-off-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-on: http://review.coreboot.org/11631 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-10-01cbfstool: Add support for hashes as file metadataPatrick Georgi
They allow optimizing a verification of a whole CBFS image by only dealing with the headers (assuming you choose to trust the hash algorithm(s)). The format allows for multiple hashes for a single file, and cbfstool can handle them, but right now it can't generate such headers. Loosely based on Sol's work in http://review.coreboot.org/#/c/10147/, but using the compatible file attribute format. vboot is now a hard dependency of the build process, but we import it into the tree for quite a while now. Change-Id: I9f14f30537d676ce209ad612e7327c6f4810b313 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11767 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-09-28cbfstool: prefer fmap data over cbfs master header if it existsPatrick Georgi
Up to now, if both fmap and a master header existed, the master header was used. Now, use the master header only if no fmap is found. Change-Id: Iafbf2c9dc325597e23a9780b495549b5d912e9ad Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: http://review.coreboot.org/11629 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-09-16cbfstool: provide metadata size to cbfs_locate_entry()Aaron Durbin
The cbfs_locate_entry() function had a hack in there which assumed a struct cbfs_stage data was being added in addition to the struct cbfs_file and name. Move that logic out to the callers while still maintaining the logic for consistency. The only impacted commands cbfs_add and cbfs_locate, but those are using the default 'always adding struct cbfs_stage' in addition to cbfs_file + name. Eventually those should be removed when cbfs_locate is removed as cbfs_add has no smarts related to the cbfs file type provided. BUG=chrome-os-partner:44827 BRANCH=None TEST=Built rambi. Change-Id: I2771116ea1ff439ea53b8886e1f33e0e637a79d4 Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/11668 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-09-14cbfstool: drop extra copy of filetype->string mapPatrick Georgi
We had two mappings of filetype IDs to strings. We shouldn't. Change-Id: I08e478b92f3316139f14294e50ede657c7d5fb01 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: http://review.coreboot.org/11626 Reviewed-by: Aaron Durbin <adurbin@chromium.org> Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2015-09-01cbfstool: implement decompression support for cbfstool extractPatrick Georgi
Change-Id: I5142b03d3c3e028eeb179f225848f762186f94a8 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11363 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-09-01cbfstool: factor out parsing compression file attributesPatrick Georgi
cbfstool extract also needs it. Change-Id: I8302bb18c5f797eb0a43ec4e4269790f3d49a896 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11361 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-09-01cbfstool: support compressed files in cbfstool printPatrick Georgi
Display compressed and decompressed sizes, as well as the compression algorithm used, when a compressed file is encountered. Change-Id: I13c2332702c4a5bec379e1ebda72753e06f8e135 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11359 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-09-01cbfstool: add extended file attributes for cbfs_filePatrick Georgi
cbfs_file_first_attr(struct cbfs_file *) and cbfs_file_next_attr(struct cbfs_file *, struct cbfs_file_attribute *) help navigate through extended attributes. cbfs_add_file_attr(header, tag, size) adds a new file attribute to header. Change-Id: I325965286c44f31abd95df684d340cebb0e68b75 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: http://review.coreboot.org/10934 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-27cbfstool: update warning on cbfstool extractPatrick Georgi
We have tons of file types now that can be safely extracted. It's pretty much only stages and payloads that aren't. Change-Id: Ibf58a2c721f863d654537850c6f93d68a8a5bbeb Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11360 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-27cbfstool: don't pass header_size as separate argument (part 2)Patrick Georgi
It's already present inside struct cbfs_file Change-Id: Ic9682e93c3d208e2ed458940e4a9f9f5a64b6e98 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11333 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-27cbfstool: don't pass header_size as separate argumentPatrick Georgi
It's already present inside struct cbfs_file Change-Id: Ib10663c6601aa02e07b868e440f05da02af9c2d9 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11332 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-27cbfstool: use struct cbfs_file * instead of void *Patrick Georgi
My concern was that compilers may something stupid under the assumption of a fixed struct size, but filename is already variable, so things are okay. Change-Id: I5348faf68f0a7993294e9de4c0b6c737278b28af Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11331 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-27cbfstool: cbfs_add_entry() doesn't need to know filename or typePatrick Georgi
They're passed as part of the header now. Change-Id: I7cd6296adac1fa72e0708b89c7009552e272f656 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11327 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-26cbfstool: move cbfs_file header creation further up the call chainPatrick Georgi
The header is now created before the "converters" are run. Adding new capabilities (and fields to the header) will happen there, so we're close. Change-Id: I0556df724bd93816b435efff7d931293dbed918f Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11326 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-26cbfstool: drop size argument to cbfs_add_entry_atPatrick Georgi
It's sole use was comparing it to the header's "len" field. Change-Id: Ic3657a709dee0d2b9288373757345a1a56124f37 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11324 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-26cbfstool: cut down on the debug outputPatrick Georgi
Change-Id: I9a0aad42e4eb67a07c939d7cfa0d2d80838412bb Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11323 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-26cbfstool: Don't patch cbfs_file->len, it's already set correctlyPatrick Georgi
->len used to be set to the file data length plus the size of the padding used for the cbfs_file header. This isn't the case anymore, so no patching of this field is necessary anymore. ->offset still needs to be patched in that case because its final value can only be determined when the file's actual location is known. Change-Id: I1037885f81b4ed3b68898dd7d0e515cf7a9c90a8 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11322 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-26cbfstool: drop unused arguments in internal functionPatrick Georgi
Change-Id: Ie4edc5f9c96ffba7dcf8b974c56851658b9538e4 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11321 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-26cbfstool: start moving cbfs_file header creation up the call chainPatrick Georgi
Up to now cbfstool creates the cbfs_file header at the latest possible time, which is unsuitable when the idea is to add further fields to it that need to be configured earlier. Thus, have it ripple up the call chain. Change-Id: I7c160681c31818bc550ed2098008146043d0ee01 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11320 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-26cbfstool: more descriptive variable namePatrick Georgi
"target", for what? It's the offset where the file header of the currently added file will be located, name it as such. Change-Id: I382f08f81991faf660e217566849773d9a7ec227 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/11319 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-13cbfstool: unify actual file creationPatrick Georgi
After the preparation in earlier commits, it is now possible to handle the more general case of position independent files using the special code path for fixed location files. This leads to a single place where non-empty cbfs file headers are actually written into the image, allowing us to move it up the chain more easily. Change-Id: I8c1fca5e4e81c20971b2960c87690e982aa3e274 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/11222 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-13cbfstool: move tests for fixed-location files earlierPatrick Georgi
... and the assert is gone. The actual action of adding a just-right file can be moved after the tests since it's exactly the condition those tests don't continue or break on. Change-Id: I6d0e829e0158198301136ada9a0de2f168ceee3f Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/11221 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-13cbfstool: prepare moving tests earlierPatrick Georgi
The assert() makes sure the if() holds true. But that assert won't survive for long. Change-Id: Iab7d2bc7bfebb3f3b3ce70dc5bd041902e14bd7a Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/11220 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-13cbfstool: factor out creating a cbfs file headerPatrick Georgi
We will want to create headers that live outside the final image at some point (eg. to build the file before we even know where to place it). Change-Id: Ie4c0323df8d5be955aec3621b75309e8f11fae49 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/11219 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-13cbfstool: Make cbfs_create_empty_entry build a more complete headerPatrick Georgi
Pass the file type into it instead of creating an entry, then modifying the header field again after the fact. Change-Id: I655583218f5085035b0f80efff7f91a66b5b296e Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/11218 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-13cbfstool: honor larger-than-usual header sizes in cbfs_add_entry_at()Patrick Georgi
If an earlier stage built a larger header, cbfs_add_entry_at() shouldn't decide to go with the most boring, least featureful header type (and its size) instead. Change-Id: Icc5dcd9a797a0f3c42f91cddd21b3b3916095b63 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/11217 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-13cbfstool: expose cbfs_calculate_file_header_size()Patrick Georgi
Headers vary in size soon, and more places need to be able to calculate their size. Change-Id: I30761bb9da0756418993dee21d8fa18cf3174c40 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/11214 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-08-13cbfstool: allow passing a precalculated header size into cbfs_add_entry()Patrick Georgi
This is in preparation of creating the cbfs_file header much earlier in the process. For now, size is enough because lots of things need to move before it makes sense to deal with cbfs_file at a higher level. Change-Id: I47589247c3011cb828170eaa10ef4a1e0f85ab84 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/11213 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-07-18cbfstool: Deduplicate code to merge empty filesPatrick Georgi
The code for removing a file had its own merge routine. Use the generic one instead. Change-Id: I90ed007ab86f78a2728f529fa0143c5c1dfbbdc3 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/10967 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-07-18cbfstool: rename checksum to attributes_offsetPatrick Georgi
So far it's still unused, but its purpose will change: It will become an offset to another structure that contains additional file attributes. This change is compatible because the binary format doesn't change and so far the field was always set to 0, which can serve nicely as 'unused' field. Change-Id: I2dafb06866713d43a236556f9492641526270837 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: http://review.coreboot.org/10933 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-07-18cbfstool: move fill value to cbfs.hPatrick Georgi
Change-Id: Ie05db6d43219c65d08e2221009875f81eb29b630 Signed-off-by: Patrick Georgi <patrick@georgi-clan.de> Reviewed-on: http://review.coreboot.org/10968 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-07-16cbfstool: Remove extra comma after {0, NULL} list elementSol Boucher
Trailing commas are useful for lists that can be extended. These lists are 0-terminated, and there should be no elements following that. Change-Id: Iea8c6d5579d6363e77e1f5af666948160c4a9bf9 Signed-off-by: Sol Boucher <solb@chromium.org> Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Change-Id: I1a117a9473e895feaf455bb30d0f945f57de51eb Original-Signed-off-by: Sol Boucher <solb@chromium.org> Reviewed-on: http://review.coreboot.org/10932 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-07-16cbfstool: Factor out compression algorithm listSol Boucher
Parse compression algorithm arguments using a single list. Change-Id: Idc5b14a53377b29964f24221e42db6e09a497d48 Signed-off-by: Sol Boucher <solb@chromium.org> Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Change-Id: I1a117a9473e895feaf455bb30d0f945f57de51eb Original-Signed-off-by: Sol Boucher <solb@chromium.org> Reviewed-on: http://review.coreboot.org/10931 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-07-16cbfstool: use variable length array to model cbfs_file's filenamePatrick Georgi
Change-Id: Ib056983630b2899d7e6cbcb43f6b7153f0f8e282 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: http://review.coreboot.org/10928 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-05-21Remove address from GPLv2 headersPatrick Georgi
As per discussion with lawyers[tm], it's not a good idea to shorten the license header too much - not for legal reasons but because there are tools that look for them, and giving them a standard pattern simplifies things. However, we got confirmation that we don't have to update every file ever added to coreboot whenever the FSF gets a new lease, but can drop the address instead. util/kconfig is excluded because that's imported code that we may want to synchronize every now and then. $ find * -type f -exec sed -i "s:Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, *MA[, ]*02110-1301[, ]*USA:Foundation, Inc.:" {} + $ find * -type f -exec sed -i "s:Foundation, Inc., 51 Franklin Street, Suite 500, Boston, MA 02110-1335, USA:Foundation, Inc.:" {} + $ find * -type f -exec sed -i "s:Foundation, Inc., 59 Temple Place[-, ]*Suite 330, Boston, MA *02111-1307[, ]*USA:Foundation, Inc.:" {} + $ find * -type f -exec sed -i "s:Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.:Foundation, Inc.:" {} + $ find * -type f -a \! -name \*.patch \ -a \! -name \*_shipped \ -a \! -name LICENSE_GPL \ -a \! -name LGPL.txt \ -a \! -name COPYING \ -a \! -name DISCLAIMER \ -exec sed -i "/Foundation, Inc./ N;s:Foundation, Inc.* USA\.* *:Foundation, Inc. :;s:Foundation, Inc. $:Foundation, Inc.:" {} + Change-Id: Icc968a5a5f3a5df8d32b940f9cdb35350654bef9 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Reviewed-on: http://review.coreboot.org/9233 Tested-by: build bot (Jenkins) Reviewed-by: Vladimir Serbinenko <phcoder@gmail.com>
2015-05-13cbfstool: Support top-aligned addresses for new-format imagesSol Boucher
The cbfstool handling of new-style FMAP-driven "partitioned" images originally disallowed the use of x86-style top-aligned addresses with the add.* and layout actions because it wasn't obvious how they should work, especially since the normal addressing is done relative to each individual region for these types of images. Not surprisingly, however, the x86 portions of the build system make copious use of top-aligned addresses, so this allows their use with new images and specifies their behavior as being relative to the *image* end---not the region end---just as it is for legacy images. Change-Id: Icecc843f4f8b6bb52aa0ea16df771faa278228d2 Signed-off-by: Sol Boucher <solb@chromium.org> Reviewed-on: http://review.coreboot.org/10136 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-05-13cbfstool: New image format w/ required FMAP and w/o CBFS master headerSol Boucher
These new-style firmware images use the FMAP of the root of knowledge about their layout, which allows them to have sections containing raw data whose offset and size can easily be determined at runtime or when modifying or flashing the image. Furthermore, they can even have multiple CBFSes, each of which occupies a different FMAP region. It is assumed that the first entry of each CBFS, including the primary one, will be located right at the start of its region. This means that the bootblock needs to be moved into its own FMAP region, but makes the CBFS master header obsolete because, with the exception of the version and alignment, all its fields are redundant once its CBFS has an entry in the FMAP. The version code will be addressed in a future commit before the new format comes into use, while the alignment will just be defined to 64 bytes in both cbfstool and coreboot itself, since there's almost no reason to ever change it in practice. The version code field and all necessary coreboot changes will come separately. BUG=chromium:470407 TEST=Build panther and nyan_big coreboot.rom and image.bin images with and without this patch, diff their hexdumps, and note that no locations differ except for those that do between subsequent builds of the same codebase. Try working with new-style images: use fmaptool to produce an FMAP section from an fmd file having raw sections and multiple CBFSes, pass the resulting file to cbfstool create -M -F, then try printing its layout and CBFSes' contents, add and remove CBFS files, and read and write raw sections. BRANCH=None Change-Id: I7dd2578d2143d0cedd652fdba5b22221fcc2184a Signed-off-by: Sol Boucher <solb@chromium.org> Original-Commit-Id: 8a670322297f83135b929a5b20ff2bd0e7d2abd3 Original-Change-Id: Ib86fb50edc66632f4e6f717909bbe4efb6c874e5 Original-Signed-off-by: Sol Boucher <solb@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/265863 Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/10135 Tested-by: build bot (Jenkins)
2015-05-08cbfstool: Restructure around support for reading/writing portions of filesSol Boucher
The buffer API that cbfstool uses to read and write files only directly supports one-shot operations on whole files. This adds an intermediate partitioned_file module that sits on top of the buffer system and has an awareness of FMAP entries. It provides an easy way to get a buffer for an individual region of a larger image file based on FMAP section name, as well as incrementally write those smaller buffers back to the backing file at the appropriate offset. The module has two distinct modes of operation: - For new images whose layout is described exclusively by an FMAP section, all the aforementioned functionality will be available. - For images in the current format, where the CBFS master header serves as the root of knowledge of the image's size and layout, the module falls back to a legacy operation mode, where it only allows manipulation of the entire image as one unit, but exposes this support through the same interface by mapping the region named SECTION_NAME_PRIMARY_CBFS ("COREBOOT") to the whole file. The tool is presently only ported onto the new module running in legacy mode: higher-level support for true "partitioned" images will be forthcoming. However, as part of this change, the crusty cbfs_image_from_file() and cbfs_image_write_file() abstractions are removed and replaced with a single cbfs_image function, cbfs_image_from_buffer(), as well as centralized image reading/writing directly in cbfstool's main() function. This reduces the boilerplate required to implement each new action, makes the create action much more similar to the others, and will make implementing additional actions and adding in support for the new format much easier. BUG=chromium:470407 TEST=Build panther and nyan_big coreboot.rom images with and without this patch and diff their hexdumps. Ensure that no differences occur at different locations from the diffs between subsequent builds of an identical source tree. Then flash a full new build onto nyan_big and watch it boot normally. BRANCH=None Change-Id: I25578c7b223bc8434c3074cb0dd8894534f8c500 Signed-off-by: Sol Boucher <solb@chromium.org> Original-Commit-Id: 7e1c96a48e7a27fc6b90289d35e6e169d5e7ad20 Original-Change-Id: Ia4a1a4c48df42b9ec2d6b9471b3a10eb7b24bb39 Original-Signed-off-by: Sol Boucher <solb@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/265581 Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/10134 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-08cbfstool: Eliminate useless cbfs_image_create() local variableSol Boucher
The only operation performed on this struct turned out to be sizeof... Change-Id: I619db60ed2e7ef6c196dd2600dc83bad2fdc6a55 Signed-off-by: Sol Boucher <solb@chromium.org> Reviewed-on: http://review.coreboot.org/10131 Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-08cbfstool: Fix leak in cbfs_image struct initializationSol Boucher
This patches a memory leak on every struct cbfs_image creation that was introduced by c1d1fd850ee7b8e52bd2ea5064fab68ac0c27098. Since that commit, the CBFS master header has been copied to a separate buffer so that its endianness could be fixed all at once; unfortunately, this buffer was malloc()'d but never free()'d. To address the issue, we replace the structure's struct cbfs_header * with a struct cbfs_header to eliminate the additional allocation. Change-Id: Ie066c6d4b80ad452b366a2a95092ed45aa55d91f Signed-off-by: Sol Boucher <solb@chromium.org> Reviewed-on: http://review.coreboot.org/10130 Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-08cbfstool: Fix cbfs_copy_instance()'s master header endiannessSol Boucher
The function hadn't been updated to account for the fact that we now copy an endianness-corrected CBFS master header into a separate buffer from the CBFS data: it still performed pointer arithmetic accross the two buffers and wrote the copied buffer into the image without restoring the original endianness. Change-Id: Ieb2a001f253494cf3a90d7e19cd260791200c4d3 Signed-off-by: Sol Boucher <solb@chromium.org> Reviewed-on: http://review.coreboot.org/10122 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-08cbfstool: Make the add action choose an aligned entries capacitySol Boucher
This fixes an inconsistency between `cbfstool create` and `cbfstool add` that was resulting in confusing claims about the amount of free space at the end of a CBFS. Calls to `cbfstool add` check whether a file fits under a given empty file entry by testing whether it would collide with the beginning of the *subsequent* file header; thus, if a file's end is unaligned, its reported size will not match the actual available capacity. Although deleted entries always end on an alignment boundary because `cbfstool remove` expands them to fill the available space, `cbfstool create` doesn't necessarily size a new entries region to result in an empty entry with an aligned end. This problem never resulted in clobbering important data because cbfstool would blindly reserve 64B (or the selected alignment) of free space immediately after the all-inclusive empty file entry. This change alters the way this reservation is reported: only the overhang past the alignment is used as hidden padding, and the empty entry's capacity is always reported such that it ends at an aligned address. Much of the time that went into this patch was spent building trust in the trickery cbfstool employs to avoid explicitly tracking the image's total capacity for entries, so below are two proofs of correctness to save others time and discourage inadvertent breakage: OBSERVATION (A): A check in cbfs_image_create() guarantees that an aligned CBFS empty file header is small enough that it won't cross another aligned address. OBSERVATION (B): In cbfs_image_create(), the initial empty entry is sized such that its contents end on an aligned address. THM. 1: Placing a new file within an empty entry located below an existing file entry will never leave an aligned flash address containing neither the beginning of a file header nor part of a file. We can prove this by contradiction: assume a newly-added file neither fills to the end of the preexisting empty entry nor leaves room for another aligned empty header after it. Then the first aligned address after the end of the newly-inserted file... - CASE 1: ...already contains a preexisting file entry header. + Then that address contains a file header. - CASE 2: ...does not already house a file entry header. + Then because CBFS content doesn't fall outside headers, the area between there and the *next* aligned address after that is unused. + By (A), we can fit a file header without clobbering anything. + Then that address now contains a file header. THM. 2: Placing a new file in an empty entry at the very end of the image such that it fits, but leaves no room for a final header, is guaranteed not to change the total amount of space for entries, even if that new file is later removed from the CBFS. Again, we use contradiction: assume that creating such a file causes a permanent... - CASE 1: ...increase in the amount of available space. + Then the combination of the inserted file, its header, and any padding must have exceeded the empty entry in size enough for it to cross at least one additional aligned address, since aligned addresses are how the limit on an entry's capacity is determined. + But adding the file couldn't have caused us to write past any further aligned addresses because they are the boundary's used when verifying that sufficient capacity exists; furthermore, by (B), no entry can ever terminate beyond where the initial empty entry did when the CBFS was first created. + Then the creation of the file did not result in a space increase. - CASE 2: ...decrease in the amount of available space. + Then the end of the new file entry crosses at least one fewer aligned address than did the empty file entry. + Then by (A), there is room to place a new file entry that describes the remaining available space at the first available aligned address. + Then there is now a new record showing the same amount of available space. + Then the creation of the file did not result in a space decrease. BUG=chromium:473726 TEST=Had the following conversation with cbfstool: $ ./cbfstool test.image create -s 0x100000 -m arm Created CBFS image (capacity = 1048408 bytes) $ ./cbfstool test.image print test.image: 1024 kB, bootblocksize 0, romsize 1048576, offset 0x40 alignment: 64 bytes, architecture: arm Name Offset Type Size (empty) 0x40 null 1048408 $ dd if=/dev/zero of=toobigmed.bin bs=1048409 count=1 1+0 records in 1+0 records out 1048409 bytes (1.0 MB) copied, 0.0057865 s, 181 MB/s $ ./cbfstool test.image add -t 0x50 -f toobigmed.bin -n toobig E: Could not add [toobigmed.bin, 1048409 bytes (1023 KB)@0x0]; too big? E: Failed to add 'toobigmed.bin' into ROM image. $ truncate -s -1 toobigmed.bin $ ./cbfstool test.image add -t 0x50 -f toobigmed.bin -n toobig $ ./cbfstool test.image print test.image: 1024 kB, bootblocksize 0, romsize 1048576, offset 0x40 alignment: 64 bytes, architecture: arm Name Offset Type Size toobig 0x40 raw 1048408 $ ./cbfstool test.image remove -n toobig $ ./cbfstool test.image print test.image: 1024 kB, bootblocksize 0, romsize 1048576, offset 0x40 alignment: 64 bytes, architecture: arm Name Offset Type Size (empty) 0x40 deleted 1048408 $ ./cbfstool test.image print test.image: 1024 kB, bootblocksize 0, romsize 1048576, offset 0x40 alignment: 64 bytes, architecture: arm Name Offset Type Size (empty) 0x40 deleted 1048408 BRANCH=None Change-Id: I118743e37469ef0226970decc900db5d9b92c5df Signed-off-by: Sol Boucher <solb@chromium.org> Original-Commit-Id: e317ddca14bc36bc36e6406b758378c88e9ae04e Original-Change-Id: I294ee489b4918646c359b06aa1581918f2d8badc Original-Signed-off-by: Sol Boucher <solb@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/263962 Original-Reviewed-by: Hung-Te Lin <hungte@chromium.org> Original-Reviewed-by: Stefan Reinauer <reinauer@google.com> Reviewed-on: http://review.coreboot.org/9939 Tested-by: build bot (Jenkins)
2015-04-28cbfstool: compare pointer difference with ptrdiff_t valuePatrick Georgi
Fixes building cbfstool in 32bit environments. Change-Id: I3c94afc9c961eb8b41d1e08f4a16e5cab2a6bb8b Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: http://review.coreboot.org/10015 Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-04-25cbfstool: Fix ability to add files at offsets near the end of empty spacesSol Boucher
Because cbfs_add_entry_at() previously *assumed* it would have to create a trailing empty entry, it was impossible to add files at exact offsets close enough to the end of an existing empty entry that they occupied the remainder of its space. This addresses the problem by skipping the step of creating the trailing empty entry if doing so would place it at the start offset of whatever already followed the original empty section. BUG=chromium:473511 TEST=Run the following commands: $ ./cbfstool test.image create -s 0x100000 -m arm $ dd if=/dev/zero of=twok.bin bs=1 count=2048 $ ./cbfstool test.image add -t 0x50 -f twok.bin -n at_end -b 0xff7c0 $ ./cbfstool test.image add -t 0x50 -f twok.bin -n near_end -b 0xfef80 $ ./cbfstool test.image print There shouldn't be any assertions, and the output should be: test.image: 1024 kB, bootblocksize 0, romsize 1048576, offset 0x40 alignment: 64 bytes, architecture: arm Name Offset Type Size (empty) 0x40 null 1044184 near_end 0xfef40 raw 2048 at_end 0xff780 raw 2048 BRANCH=None Change-Id: Ic8a6c3dfa4f82346a067c0804afb6c5a5e89e6c8 Signed-off-by: Sol Boucher <solb@chromium.org> Original-Commit-Id: 1bbd353fddc818f725e488e8f2fb6e967033539d Original-Change-Id: I15d25df80787a8e34c2237262681720203509c72 Original-Signed-off-by: Sol Boucher <solb@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/263809 Original-Reviewed-by: Hung-Te Lin <hungte@chromium.org> Original-Reviewed-by: Stefan Reinauer <reinauer@google.com> Reviewed-on: http://review.coreboot.org/9938 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com> Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2015-04-25cbfstool: Clean up in preparation for adding new filesSol Boucher
This enables more warnings on the cbfstool codebase and fixes the issues that surface as a result. A memory leak that used to occur when compressing files with lzma is also found and fixed. Finally, there are several fixes for the Makefile: - Its autodependencies used to be broken because the target for the .dependencies file was misnamed; this meant that Make didn't know how to rebuild the file, and so would silently skip the step of updating it before including it. - The ability to build to a custom output directory by defining the obj variable had bitrotted. - The default value of the obj variable was causing implicit rules not to apply when specifying a file as a target without providing a custom value for obj. - Add a distclean target for removing the .dependencies file. BUG=chromium:461875 TEST=Build an image with cbfstool both before and after. BRANCH=None Change-Id: I951919d63443f2b053c2e67c1ac9872abc0a43ca Signed-off-by: Sol Boucher <solb@chromium.org> Original-Commit-Id: 49293443b4e565ca48d284e9a66f80c9c213975d Original-Change-Id: Ia7350c2c3306905984cfa711d5fc4631f0b43d5b Original-Signed-off-by: Sol Boucher <solb@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/257340 Original-Reviewed-by: Julius Werner <jwerner@chromium.org> Original-Reviewed-by: Stefan Reinauer <reinauer@chromium.org> Reviewed-on: http://review.coreboot.org/9937 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-04-18cbfstool: clean up source codeVadim Bendebury
The following changes were made: - order commands and options definitions alphabetically - do not report errors at cbfs_image_from_file() call sites - the error is reported by the function itself - remove the unused parameter in cbfs_create_empty_entry() prototype BRANCH=storm BUG=none TEST=compiled cbfstool, built a storm image, observed that the image still boots Change-Id: I31b15fab0a63749c6f2d351901ed545de531eb39 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: a909a50e03be77f972b1a497198fe758661aa9f8 Original-Change-Id: I4b8898dbd44eeb2c6b388a485366e4e22b1bed16 Original-Signed-off-by: Vadim Bendebury <vbendeb@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/237560 Original-Reviewed-by: David Hendricks <dhendrix@chromium.org> Reviewed-on: http://review.coreboot.org/9746 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-04-18cbfstool: add a command to duplicate a cbfs instanceVadim Bendebury
The new command allows to create a file where the original CBFS image is duplicated at a different offset. The required options of the new command are -D, the offset where the copy CBFS header is placed, and -s, the size of the new CBFS copy. When a CBFS is copied, the bootblock area of the source CBFS is ignored, as well as empty and deleted files in the source CBFS. The size of the destination CBFS is calculated as the rombase size of the source CBFS less the bootblock size. The copy instance can be created in the image only above the original, which rules out the use of this new command for x86 images. If necessary, this limitation could be addressed later. As with other cbfstool commands, unless explicitly specified the lowest CBFS instance in the image is considered the source. If necessary, the user can specify the source CBFS using the -H option. BRANCH=storm BUG=chrome-os-partner:34161, chromium:445938 TEST=run multiple cbfstool commands on a storm image: $ cd /tmp $ cp /build/storm/firmware/image.serial.bin storm.bin $ cbfstool storm.bin print storm.bin: 8192 kB, bootblocksize 34472, romsize 458752, offset 0x8700 alignment: 64 bytes, architecture: arm Name Offset Type Size cdt.mbn 0x8700 raw 416 ddr.mbn 0x8900 raw 25836 rpm.mbn 0xee40 raw 78576 tz.mbn 0x22180 raw 85360 fallback/verstage 0x36f40 stage 41620 fallback/romstage 0x41240 stage 19556 fallback/ramstage 0x45f00 stage 25579 config 0x4c340 raw 2878 fallback/payload 0x4cec0 payload 64811 u-boot.dtb 0x5cc40 (unknown) 2993 (empty) 0x5d840 null 75608 $ cbfstool storm.bin copy -D 0x420000 E: You need to specify -s/--size. $ cbfstool storm.bin copy -D 0x420000 -s 0x70000 $ cbfstool storm.bin print W: Multiple (2) CBFS headers found, using the first one. storm.bin: 8192 kB, bootblocksize 34472, romsize 458752, offset 0x8700 alignment: 64 bytes, architecture: arm Name Offset Type Size cdt.mbn 0x8700 raw 416 ddr.mbn 0x8900 raw 25836 rpm.mbn 0xee40 raw 78576 tz.mbn 0x22180 raw 85360 fallback/verstage 0x36f40 stage 41620 fallback/romstage 0x41240 stage 19556 fallback/ramstage 0x45f00 stage 25579 config 0x4c340 raw 2878 fallback/payload 0x4cec0 payload 64811 u-boot.dtb 0x5cc40 (unknown) 2993 (empty) 0x5d840 null 75608 cbfstool storm.bin print -H 0x420000 storm.bin: 8192 kB, bootblocksize 0, romsize 4784128, offset 0x420040 alignment: 64 bytes, architecture: arm Name Offset Type Size cdt.mbn 0x420040 raw 416 ddr.mbn 0x420240 raw 25836 rpm.mbn 0x426780 raw 78576 tz.mbn 0x439ac0 raw 85360 fallback/verstage 0x44e880 stage 41620 fallback/romstage 0x458b80 stage 19556 fallback/ramstage 0x45d840 stage 25579 config 0x463c80 raw 2878 fallback/payload 0x464800 payload 64811 u-boot.dtb 0x474580 (unknown) 2993 (empty) 0x475180 null 110168 $ cbfstool storm.bin remove -n config -H 0x420000 $ cbfstool storm.bin copy -H 0x420000 -D 0x620000 -s 0x70000 $ cbfstool storm.bin print -H 0x620000 storm.bin: 8192 kB, bootblocksize 0, romsize 6881280, offset 0x620040 alignment: 64 bytes, architecture: arm Name Offset Type Size cdt.mbn 0x620040 raw 416 ddr.mbn 0x620240 raw 25836 rpm.mbn 0x626780 raw 78576 tz.mbn 0x639ac0 raw 85360 fallback/verstage 0x64e880 stage 41620 fallback/romstage 0x658b80 stage 19556 fallback/ramstage 0x65d840 stage 25579 fallback/payload 0x663c80 payload 64811 u-boot.dtb 0x673a00 (unknown) 2993 (empty) 0x674600 null 113112 $ cbfstool /build/storm/firmware/image.serial.bin extract -n fallback/payload -f payload1 [..] $ cbfstool storm.bin extract -H 0x620000 -n fallback/payload -f payload2 [..] $ diff payload1 payload2 Change-Id: Ieb9205848aec361bb870de0d284dff06c597564f Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: b8d3c1b09a47ca24d2d2effc6de0e89d1b0a8903 Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org> Original-Signed-off-by: Vadim Bendebury <vbendeb@chromium.org> Original-Change-Id: I227e607ccf7a9a8e2a1f3c6bbc506b8d29a35b1b Original-Reviewed-on: https://chromium-review.googlesource.com/237561 Reviewed-on: http://review.coreboot.org/9742 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-04-18cbfstool: allow user to explicitly specify header locationVadim Bendebury
There potentially could be multiple CBFS instances present in the firmware image. cbfstool should be able to operate on any of them, not just the first one present. To accomplish that, allow all CBFS commands to accept the -H parameter (which specifies the exact CBFS header location in the image). If this parameter is specified, the image is not searched for the CBFS header, only the specified location is checked for validity, If the location is valid, it is considered to be the CBFS header, if not - the tool exits with an error status. Note, that default behavior of the tool does not change. BRANCH=storm BUG=chrome-os-partner:34161, chromium:445938 TEST=run the following experiments: - examined an image with three CBFS instances, was able to print all of them. - built a rambi coreboot image and tried the following (cbfstool output abbreviated): $ ./util/cbfstool/cbfstool /build/rambi/firmware/coreboot.rom print coreboot.rom: 8192 kB, bootblocksize 2448, romsize 8388608, offset 0x700000 alignment: 64 bytes, architecture: x86 Name Offset Type Size cmos_layout.bin 0x700000 cmos_layout 1164 ... (empty) 0x7ec600 null 77848 $ \od -tx4 -Ax /build/rambi/firmware/coreboot.rom | tail -2 7ffff0 fff67de9 000000ff fff6dfe9 fffff650 800000 $ ./util/cbfstool/cbfstool /build/rambi/firmware/coreboot.rom print -H 0x7ff650 coreboot.rom: 8192 kB, bootblocksize 2448, romsize 8388608, offset 0x700000 alignment: 64 bytes, architecture: x86 Name Offset Type Size cmos_layout.bin 0x700000 cmos_layout 1164 ... (empty) 0x7ec600 null 77848 $ ./util/cbfstool/cbfstool /build/rambi/firmware/coreboot.rom print -H 0x7ff654 E: /build/rambi/firmware/coreboot.rom does not have CBFS master header. E: Could not load ROM image '/build/rambi/firmware/coreboot.rom'. $ Change-Id: I64cbdc79096f3c7a113762b641305542af7bbd60 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: 86b88222df6eed25bb176d653305e2e57e18b73a Original-Change-Id: I486092e222c96c65868ae7d41a9e8976ffcc93c4 Original-Signed-off-by: Vadim Bendebury <vbendeb@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/237485 Original-Reviewed-by: David Hendricks <dhendrix@chromium.org> Original-Reviewed-by: Patrick Georgi <pgeorgi@chromium.org> Original-Reviewed-by: Stefan Reinauer <reinauer@google.com> Reviewed-on: http://review.coreboot.org/9741 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-04-14CBFS: Automate ROM image layout and remove hardcoded offsetsJulius Werner
Non-x86 boards currently need to hardcode the position of their CBFS master header in a Kconfig. This is very brittle because it is usually put in between the bootblock and the first CBFS entry, without any checks to guarantee that it won't overlap either of those. It is not fun to debug random failures that move and disappear with tiny alignment changes because someone decided to write "ORBC1112" over some part of your data section (in a way that is not visible in the symbolized .elf binaries, only in the final image). This patch seeks to prevent those issues and reduce the need for manual configuration by making the image layout a completely automated part of cbfstool. Since automated placement of the CBFS header means we can no longer hardcode its position into coreboot, this patch takes the existing x86 solution of placing a pointer to the header at the very end of the CBFS-managed section of the ROM and generalizes it to all architectures. This is now even possible with the read-only/read-write split in ChromeOS, since coreboot knows how large that section is from the CBFS_SIZE Kconfig (which is by default equal to ROM_SIZE, but can be changed on systems that place other data next to coreboot/CBFS in ROM). Also adds a feature to cbfstool that makes the -B (bootblock file name) argument on image creation optional, since we have recently found valid use cases for CBFS images that are not the first boot medium of the device (instead opened by an earlier bootloader that can already interpret CBFS) and therefore don't really need a bootblock. BRANCH=None BUG=None TEST=Built and booted on Veyron_Pinky, Nyan_Blaze and Falco. Change-Id: Ib715bb8db258e602991b34f994750a2d3e2d5adf Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: e9879c0fbd57f105254c54bacb3e592acdcad35c Original-Change-Id: Ifcc755326832755cfbccd6f0a12104cba28a20af Original-Signed-off-by: Julius Werner <jwerner@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/229975 Reviewed-on: http://review.coreboot.org/9620 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-03-30cbfstool: add filetypes for FSP, MRC, SPC and MRC_CACHEMartin Roth
This adds a few new file types to cbfstool. Currently these files are being added using bare hex values in the coreboot makefiles. This patch is just to make the values official and to help get rid of some confusion in the values used within the makefiles. All of these new types are roughly equivalent to raw. Change-Id: I37c4180a247136cd98080f6f7609d3cf905a62f5 Signed-off-by: Martin Roth <gaumless@gmail.com> Reviewed-on: http://review.coreboot.org/8977 Tested-by: build bot (Jenkins) Reviewed-by: Marc Jones <marc.jones@se-eng.com> Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Aaron Durbin <adurbin@google.com>
2015-03-04cbfstool: Clean up codeStefan Reinauer
cbfstool has diverged between coreboot upstream and the chromium tree. Bring in some of the chromium changes, in particular the useful remainders of cbf37fe (https://chromium-review.googlesource.com/176710) - fix coding style - mark unused variables explicitly unused - remove some dead code Change-Id: I354aaede8ce425ebe99d4c60c232feea62bf8a11 Signed-off-by: Stefan Reinauer <reinauer@google.com> Reviewed-on: http://review.coreboot.org/8577 Tested-by: build bot (Jenkins) Reviewed-by: Marc Jones <marc.jones@se-eng.com>
2015-01-03util/cbfstool: Fix byte-ordering for payload type field.Hung-Te Lin
In https://chromium-review.googlesource.com/181272 the payload->type has been changed to big-endian (network ordering) but the cbfs_image is still parsing type as host ordering, which caused printing cbfs image verbosely (cbfstool imge print -v) to fail to find entry field and print numerous garbage output. Payload fields should be always parsed in big-endian (network ordering). BUG=none TEST=make; cbfstool image.bin print -v -v -v # see payloads correctly Original-Change-Id: If1ac355b8847fb54988069f694bd2f317ce49a1a Original-Signed-off-by: Hung-Te Lin <hungte@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/200158 Original-Reviewed-by: Ronald Minnich <rminnich@chromium.org> (cherry picked from commit 423f7dd28f8b071692d57401e144232d5ee2e479) Signed-off-by: Marc Jones <marc.jones@se-eng.com> Change-Id: I5a4694e887c7ff48d8d0713bb5808c29256141a9 Reviewed-on: http://review.coreboot.org/8005 Reviewed-by: Ronald G. Minnich <rminnich@gmail.com> Tested-by: build bot (Jenkins)
2014-09-12cbfstool: Fix architecture check when adding payloadStefan Reinauer
In the process of rewriting cbfstool for ARM and using a new internal API a regression was introduced that would silently let you add an ARM payload into an x86 CBFS image and the other way around. This patch fixes cbfstool to produce an error in that case again. Change-Id: I37ee65a467d9658d0846c2cf43b582e285f1a8f8 Signed-off-by: Stefan Reinauer <reinauer@google.com> Reviewed-on: https://chromium-review.googlesource.com/176711 Reviewed-by: Hung-Te Lin <hungte@chromium.org> Commit-Queue: Stefan Reinauer <reinauer@chromium.org> Tested-by: Stefan Reinauer <reinauer@chromium.org> (cherry picked from commit 8f74f3f5227e440ae46b59f8fd692f679f3ada2d) Signed-off-by: Isaac Christensen <isaac.christensen@se-eng.com> Reviewed-on: http://review.coreboot.org/6879 Tested-by: build bot (Jenkins) Reviewed-by: Edward O'Callaghan <eocallaghan@alterapraxis.com> Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2014-08-07cbfstool: process cbfs_payload_segment(s) in host byte orderAaron Durbin
The printing routines of the cbfs_payload_segment assumed the type could be accessed in host order. Each of the fields need to be converted to the host order before inspecting the fields. In addition, this removes all the ntoh*() calls while processing the cbfs_payload_segment structures. cbfstool would crash adding entries or just printing entries containing a payload when -v was passed on the command line. Change-Id: Iff41c64a99001b9e3920e2e26828c5fd6e671239 Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/6498 Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Martin Roth <gaumless@gmail.com>
2014-04-07util/cbfstool: Make cbfs_image_delete() NULL-tolerant.Edward O'Callaghan
This fixes a double free crash that occurs when a call to cbfs_image_from_file() fails in cbfs_extract() and falls though to cbfs_image_delete() with a NULL-pointer. To reproduce the crash pass the following arguments where the files passed, in fact, do not exist. As follows: ./cbfstool build/coreboot.rom extract -n config -f /tmp/config.txt Change-Id: I2213ff175d0703705a0ec10271b30bb26b6f8d0a Signed-off-by: Edward O'Callaghan <eocallaghan@alterapraxis.com> Reviewed-on: http://review.coreboot.org/5353 Tested-by: build bot (Jenkins) Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
2014-03-13cbfstool: add eflparsing.hAaron Durbin
elfparsing.h serves as the header to working with the elf parser. Additionally, only include what is needed by the other files. Many had no reason to be including elf.h aside from fixing compilation problems when including cbfs.h. Change-Id: I9eb5f09f3122aa18beeca52d2e4dc2102d70fb9d Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/5370 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org> Reviewed-by: Marc Jones <marc.jones@se-eng.com>
2014-02-11cbfstool: Deserialize CBFS master header when reading imageAlexandru Gagniuc
Rather than using [hn]to[nh] whenever accessing a member of the CBFS header, deserialize the header when opening the CBFS image. The header is no longer a pointer inside the CBFS buffer, but a separate struct, a copy of the original header in a host-friendly format. This kills more of the ntohl usage. Change-Id: I5f8a5818b9d5a2d1152b1906249c4a5847d02bac Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Reviewed-on: http://review.coreboot.org/5121 Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2014-02-02cbfstool: add code to serialize the header using the new xdr functionsRonald G. Minnich
This change adds a header serialization function. Programmers can thus just set up a header as needed, without worrying about forgetting if and how to use the [hn]to[hn]* functions. In the long term, we will work to remove swab.h, i.e. we need to get to the point where programmers don't have to try to remember [hn]to[nh]* and where it goes. To date, even the best programmers we have have made an error with those functions, and those errors have persisted for 6 or 7 years now. It's very easy to make that mistake. BUG=None TEST=Build a peppy image and verify that it's bit for bit the same. All chromebooks use this code and build and boot correctly. BRANCH=None Change-Id: I0f9b8e7cac5f52d0ea330ba948650fa0803aa0d5 Signed-off-by: Ronald G. Minnich <rminnich@google.com> Reviewed-on: https://chromium-review.googlesource.com/181552 Reviewed-by: Ronald Minnich <rminnich@chromium.org> Commit-Queue: Ronald Minnich <rminnich@chromium.org> Tested-by: Ronald Minnich <rminnich@chromium.org> Reviewed-on: http://review.coreboot.org/5100 Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2014-02-02Add section header parsing and use it in the mk-payload stepRonald G. Minnich
This completes the improvements to the ELF file parsing code. We can now parse section headers too, across all 4 combinations of word size and endianness. I had hoped to completely remove the use of htonl until I found it in cbfs_image.c. That's a battle for another day. There's now a handy macro to create magic numbers in host byte order. I'm using it for all the PAYLOAD_SEGMENT_* constants and maybe we can use it for the others too, but this is sensitive code and I'd rather change one thing at a time. To maximize the ease of use for users, elf parsing is accomplished with just one function: int elf_headers(const struct buffer *pinput, Elf64_Ehdr *ehdr, Elf64_Phdr **pphdr, Elf64_Shdr **pshdr) which requires the ehdr and pphdr pointers to be non-NULL, but allows the pshdr to be NULL. If pshdr is NULL, the code will not try to read in section headers. To satisfy our powerful scripts, I had to remove the ^M from an unrelated microcode file. BUG=None TEST=Build a peppy image (known to boot) with old and new versions and verify they are bit-for-bit the same. This was also fully tested across all chromebooks for building and booting and running chromeos. BRANCH=None Change-Id: I54dad887d922428b6175fdb6a9cdfadd8a6bb889 Signed-off-by: Ronald G. Minnich <rminnich@google.com> Reviewed-on: https://chromium-review.googlesource.com/181272 Reviewed-by: Ronald Minnich <rminnich@chromium.org> Commit-Queue: Ronald Minnich <rminnich@chromium.org> Tested-by: Ronald Minnich <rminnich@chromium.org> Signed-off-by: Ronald G. Minnich <rminnich@google.com> Reviewed-on: http://review.coreboot.org/5098 Tested-by: build bot (Jenkins) Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
2014-01-29cbfs: fix issues with word size and endianness.Ronald G. Minnich
Add XDR functions and use them to convert the ELF headers to native headers, using the Elf64 structs to ensure we accomodate all word sizes. Also, use these XDR functions for output. This may seem overly complex but it turned out to be much the easiest way to do this. Note that the basic elf parsing function in cbfs-mkstage.c now works over all ELF files, for all architectures, endian, and word size combinations. At the same time, the basic elf parsing in cbfs-mkstage.c is a loop that has no architecture-specific conditionals. Add -g to the LDFLAGS while we're here. It's on the CFLAGS so there is no harm done. This code has been tested on all chromebooks that use coreboot to date. I added most of the extra checks from ChromeOS and they triggered a lot of warnings, hence the other changes. I had to take -Wshadow back out due to the many errors it triggers in LZMA. BUG=None TEST=Build and boot for Peppy; works fine. Build and boot for nyan, works fine. Build for qemu targets and armv8 targets. BRANCH=None Change-Id: I5a4cee9854799189115ac701e22efc406a8d902f Signed-off-by: Ronald G. Minnich <rminnich@google.com> Reviewed-on: https://chromium-review.googlesource.com/178606 Reviewed-by: Ronald Minnich <rminnich@chromium.org> Commit-Queue: Ronald Minnich <rminnich@chromium.org> Tested-by: Ronald Minnich <rminnich@chromium.org> Reviewed-on: http://review.coreboot.org/4817 Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Tested-by: build bot (Jenkins) Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2014-01-24cbfstool: correct size left calculation for "empty" entriesAaron Durbin
After removing a file sandwiched between two other files, that file could no longer be re-added at the same location. cbfstool tried to add the file, and a new "empty" entry, which, together, would no longer fit, so it continued checking for the next available space. Change the behavior to add the file if there is enough space for the file alone, then only add the "empty" entry if there is enough space for it. Change-Id: Iad3897dd28cf12f12ae877cfd83e1990fa7d2f0f Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Signed-off-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/4772 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@google.com>
2013-08-16Correct spelling of shadow, setting and memoryPaul Menzel
Change-Id: Ic7d793754a8b59623b49b7a88c09b5c6b6ef2cf0 Signed-off-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-on: http://review.coreboot.org/3768 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <patrick@georgi-clan.de>
2013-03-27cbfstool: Fix cbfs_image.cStefan Reinauer
- The read-only structures are const now - cosmetic fixes - put { on a new line for functions - move code after structures Change-Id: Ib9131b80242b91bd5105feaebdf8306a844da1cc Signed-off-by: Stefan Reinauer <reinauer@google.com> Reviewed-on: http://review.coreboot.org/2922 Reviewed-by: Aaron Durbin <adurbin@google.com> Tested-by: build bot (Jenkins)
2013-03-22cbfstool: Fix initial empty space in image creation.Hung-Te Lin
When calculating initial CBFS empty entry space, the size of header itself must be not included (with the reserved space for entry name). This is a regression of the old cbfstool size bug. Before this fix, in build process we see: OBJCOPY cbfs/fallback/romstage_null.bin W: CBFS image was created with old cbfstool with size bug. Fixing size in last entry... And checking the output binary: cbfstool build/coreboot.pre1 print -v -v DEBUG: read_cbfs_image: build/coreboot.pre1 (262144 bytes) DEBUG: x86sig: 0xfffffd30, offset: 0x3fd30 W: CBFS image was created with old cbfstool with size bug. Fixing size in last entry... DEBUG: Last entry has been changed from 0x3fd40 to 0x3fd00. coreboot.pre1: 256 kB, bootblksz 688, romsize 262144, offset 0x0 align: 64 Name Offset Type Size (empty) 0x0 null 261296 DEBUG: cbfs_file=0x0, offset=0x28, content_address=0x28+0x3fcb0 After this fix, no more alerts in build process. Verified to build successfully on x86/qemu and arm/snow configurations. Change-Id: I35c96f4c10a41bae671148a0e08988fa3bf6b7d3 Signed-off-by: Hung-Te Lin <hungte@chromium.org> Reviewed-on: http://review.coreboot.org/2731 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2013-03-20cbfstool locate: Implement alignment switch --align/-aHung-Te Lin
cbfstool usage change: "-a" for "cbfstool locate" can specify base address alignment. To support putting a blob in aligned location (ex, microcode needs to be aligned in 0x10), alignment (-a) is implemented into "locate" command. Verified by manually testing a file (324 bytes) with alignment=0x10: cbfstool coreboot.rom locate -f test -n test -a 0x10 # output: 0x71fdd0 cbfstool coreboot.rom add -f test -n test -t raw -b 0x71fdd0 cbfstool coreboot.rom print -v -v # output: test 0x71fd80 raw 324 # output: cbfs_file=0x71fd80, offset=0x50, content_address=0x71fdd0+0x144 Also verified to be compatible with old behavior by building i386/axus/tc320 (with page limitation 0x40000): cbfstool coreboot.rom locate -f romstage_null.bin -n romstage -P 0x40000 # output: 0x44 cbfstool coreboot.rom locate -f x.bin -n romstage -P 0x40000 -a 0x30 # output: 0x60 Change-Id: I78b549fe6097ce5cb6162b09f064853827069637 Signed-off-by: Hung-Te Lin <hungte@chromium.org> Reviewed-on: http://review.coreboot.org/2824 Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Tested-by: build bot (Jenkins)