| Age | Commit message (Collapse) | Author |
|---|
|
When we first created the arm64 port, we weren't quite sure whether
coreboot would always run in EL3 on all platforms. The AArch64 A.R.M.
technically considers this exception level optional, but in practice all
SoCs seem to support it. We have since accumulated a lot of code that
already hardcodes an implicit or explicit assumption of executing in EL3
somewhere, so coreboot wouldn't work on a system that tries to enter it
in EL1/2 right now anyway.
However, some of our low level support libraries (in particular those
for accessing architectural registers) still have provisions for
running at different exception levels built-in, and often use switch
statements over the current exception level to decide which register to
access. This includes an unnecessarily large amount of code for what
should be single-instruction operations and precludes further
optimization via inlining.
This patch removes any remaining code that dynamically depends on the
current exception level and makes the assumption that coreboot executes
at EL3 official. If this ever needs to change for a future platform, it
would probably be cleaner to set the expected exception level in a
Kconfig rather than always probing it at runtime.
Change-Id: I1a9fb9b4227bd15a013080d1c7eabd48515fdb67
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/27880
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Patrick Rudolph <patrick.rudolph@9elements.com>
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
|
|
Disabling the MMU with proper cache behavior is a bit tricky on ARM64:
you can flush the cache first and then disable the MMU (like we have
been doing), but then you run the risk of having new cache lines
allocated in the tiny window between the two, which may or may not
become a problem when those get flushed at a later point (on some
platforms certain memory regions "go away" at certain points in a way
that makes the CPU very unhappy if it ever issues a write cycle to
them again afterwards).
The obvious alternative is to first disable the MMU and then flush the
cache, ensuring that every memory access after the flush already has the
non-cacheable attribute. But we can't just flip the order around in the
C code that we have because then those accesses in the tiny window
in-between will go straight to memory, so loads may yield the wrong
result or stores may get overwritten again by the later cache flush.
In the end, this all shouldn't really be a problem because we can do
both operations purely from registers without doing any explicit memory
accesses in-between. We just have to reimplement the function in
assembly to make sure the compiler doesn't insert any stack accesses at
the wrong points.
Change-Id: Ic552960c91400dadae6f130b2521a696eeb4c0b1
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/27238
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
|
|
New API required by sdm845 DDR init/training protocol
TEST=build & run
Change-Id: I8442442c0588dd6fb5e461b399e48a761f7bbf29
Signed-off-by: T Michael Turney <mturney@codeaurora.org>
Reviewed-on: https://review.coreboot.org/25818
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
|
|
The VA space needs to be extended to support 48bit, as on Cavium SoCs
the MMIO starts at 1 << 47.
The following changes were done to coreboot and libpayload:
* Use page table lvl 0
* Increase VA bits to 48
* Enable 256TB in MMU controller
* Add additional asserts
Tested on Cavium SoC and two ARM64 Chromebooks.
Change-Id: I89e6a4809b6b725c3945bad7fce82b0dfee7c262
Signed-off-by: Patrick Rudolph <patrick.rudolph@9elements.com>
Reviewed-on: https://review.coreboot.org/24970
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
|
|
coreboot and libpayload currently use completely different code to
perform a full cache flush on ARM64, with even different function names.
The libpayload code is closely inspired by the ARM32 version, so for the
sake of overall consistency let's sync coreboot to that. Also align a
few other cache management details to work the same way as the
corresponding ARM32 parts (such as only flushing but not invalidating
the data cache after loading a new stage, which may have a small
performance benefit).
Change-Id: I9e05b425eeeaa27a447b37f98c0928fed3f74340
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/19785
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
|
|
On ARM64, the memory type for accessing page table descriptors during
address translation is governed by the Translation Control Register
(TCR). When the MMU code accesses the same descriptors to change page
mappings, it uses the standard memory type rules (defined by the page
table descriptor for the page that contains that table, or 'device' if
the MMU is off).
Accessing the same memory with different memory types can lead to all
kinds of fun and hard to debug effects. In particular, if the TCR says
"cacheable" and the page tables say "uncacheable", page table walks will
pull stale entries into the cache and later mmu_config_range() calls
will write directly to memory, bypassing those cache lines. This means
the translations will not get updated even after a TLB flush, and later
cache flushes/evictions may write the stale entries back to memory.
Since page table configuration is currently always done from SoC code,
we can't generally ensure that the TTB is always mapped as cacheable.
We can however save developers of future SoCs a lot of headaches and
time by spot checking the attributes when the MMU gets enabled, as this
patch does.
BRANCH=None
BUG=None
TEST=Booted Oak. Manually tested get_pte() with a few addresses.
Change-Id: I3afd29dece848c4b5f759ce2f00ca2b7433374da
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: f3947f4bb0abf4466006d5e3a962bbcb8919b12d
Original-Change-Id: I1008883e5ed4cc37d30cae5777a60287d3d01af0
Original-Signed-off-by: Julius Werner <jwerner@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/323862
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: https://review.coreboot.org/13595
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
In order to have a proper runtime-modifyable page table API (e.g. to
remap DRAM after it was intialized), we need to remove any external
bookkeeping kept in global variables (which do not persist across
stages) from the MMU code. This patch implements this in a similar way
as it has recently been done for ARM32 (marking free table slots with a
special sentinel value in the first PTE that cannot occur as part of a
normal page table).
Since this requires the page table buffer to be known at compile-time,
we have to remove the option of passing it to mmu_init() at runtime
(which I already kinda deprecated before). The existing Tegra chipsets
that still used it are switched to instead define it in memlayout in a
minimally invasive change. This might not be the best way to design this
overall (I think we should probably just throw the tables into SRAM like
on all other platforms), but I don't have a Tegra system to test so I'd
rather keep this change low impact and leave the major redesign for
later.
Also inlined some single-use one-liner functions in mmu.c that I felt
confused things more than they cleared up, and fixed an (apparently
harmless?) issue with forgetting to mask out the XN page attribute bit
when casting a table descriptor to a pointer.
BRANCH=None
BUG=None
TEST=Compiled Ryu and Smaug. Booted Oak.
Change-Id: Iad71f97f5ec4b1fc981dbc8ff1dc88d96c8ee55a
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: http://review.coreboot.org/12075
Tested-by: build bot (Jenkins)
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
|
|
Set XN bit of block upper attribute to device memory in mmu. CPU may
speculatively prefetch instructions from device memory, but the IO
subsystem of some implementation may not support this operation. Set
this attribute to device memory mmu entries can prevent CPU from
prefetching device memory.
BRANCH=none
BUG=none
TEST=build and booted to kernel on oak-rev3 with dcm enabled.
Change-Id: I52ac7d7c84220624aaf6a48d64b9110d7afeb293
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 7b01a4157cb046a5e75ea7625060a602e7a63c3c
Original-Change-Id: Id535e990a23b6c89123b5a4e64d7ed21eebed607
Original-Signed-off-by: Jimmy Huang <jimmy.huang@mediatek.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/302301
Original-Commit-Ready: Yidi Lin <yidi.lin@mediatek.com>
Original-Tested-by: Yidi Lin <yidi.lin@mediatek.com>
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Reviewed-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: http://review.coreboot.org/11722
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
|
|
BRANCH=none
BUG=none
TEST=Booted on Oak and confirmed the output cacheable/non-cacheable
string is correct.
Change-Id: I062c1cc384b8cb9d07038399b1bc7ef47d992103
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 45552f95b55cd9fc81817e4ff02c78e885377065
Original-Change-Id: Ie52066dbefd2f54d0746792b89f0b57767811adb
Original-Signed-off-by: Jimmy Huang <jimmy.huang@mediatek.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/273994
Original-Reviewed-by: Julius Werner <jwerner@chromium.org>
Original-Commit-Queue: Yidi Lin <yidi.lin@mediatek.com>
Original-Tested-by: Yidi Lin <yidi.lin@mediatek.com>
Reviewed-on: http://review.coreboot.org/10390
Tested-by: build bot (Jenkins)
Reviewed-by: Furquan Shaikh <furquan@google.com>
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
We have observed issues with enabling CONFIG_SMP and adding prints
before MMU is enabled on Tegra-based SoCs. This seems to be related
to the hardware assisted locks and the restrictions laid down by ARMv8
spec.
BUG=None
BRANCH=None
TEST=Boots to kernel prompt on smaug.
Change-Id: I29a52f5a972baf396c01faba3ae3e5ecd27563e9
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: f52ee4b5b2e9b7f54eee0d105cb7e17f9a7e1613
Original-Change-Id: I432895560f468903c7beef00e78b6d38275a619c
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/272449
Original-Trybot-Ready: Furquan Shaikh <furquan@chromium.org>
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/10311
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Furquan Shaikh <furquan@google.com>
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
The current arm64 MMU interface is difficult to use in pre-RAM
environments. It is based on the memranges API which makes use of
malloc(), and early stages usually don't have a heap. It is also built
as a one-shot interface that requires all memory ranges to be laid out
beforehand, which is a problem when existing areas need to change (e.g.
after initializing DRAM).
The long-term goal of this patch is to completely switch to a
configure-as-you-go interface based on the mmu_config_range() function,
similar to what ARM32 does. As a first step this feature is added
side-by-side to the existing interface so that existing SoC
implementations continue to work and can be slowly ported over one by
one. Like the ARM32 version it does not garbage collect page tables that
become unused, so repeated mapping at different granularities will
exhaust the available table space (this is presumed to be a reasonable
limitation for a firmware environment and keeps the code much simpler).
Also do some cleanup, align comments between coreboot and libpayload for
easier diffing, and change all error cases to assert()s. Right now the
code just propagates error codes up the stack until it eventually
reaches a function that doesn't check them anymore. MMU configuration
errors (essentially just misaligned requests and running out of table
space) should always be compile-time programming errors, so failing hard
and fast seems like the best way to deal with them.
BRANCH=None
BUG=None
TEST=Compile-tested rush_ryu. Booted on Oak and hacked MMU init to use
mmu_config_range() insted of memranges. Confirmed that CRCs over all page
tables before and after the change are equal.
Change-Id: I93585b44a277c1d96d31ee9c3dd2522b5e10085b
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: f10fcba107aba1f3ea239471cb5a4f9239809539
Original-Change-Id: I6a2a11e3b94e6ae9e1553871f0cccd3b556b3e65
Original-Signed-off-by: Julius Werner <jwerner@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/271991
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/10304
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
1. change mmu granule size from 64KB to 4KB
2. correct level 1 translation table creation logic
3. automatically calculate granule size related macros
BRANCH=none
BUG=none
TEST=boot to kernel on oak board
Change-Id: I9e99a3017033f6870b1735ac8faabb267c7be0a4
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 2f18c4d5d9902f2830db82720c5543af270a7e3c
Original-Change-Id: Ia27a414ab7578d70b00c36f9c063983397ba7927
Original-Signed-off-by: Jimmy Huang <jimmy.huang@mediatek.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/265603
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Commit-Queue: Yidi Lin <yidi.lin@mediatek.com>
Original-Tested-by: Yidi Lin <yidi.lin@mediatek.com>
Reviewed-on: http://review.coreboot.org/10009
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Tested-by: build bot (Jenkins)
|
|
In order to allow proper working of caches, set the correct
shareability option for normal memory.
BUG=chrome-os-partner:38222
BRANCH=None
TEST=Compiles successfully for foster and SMP works.
Change-Id: I5462cb0a2ff94a854f71f58709d7b2e8297ccc44
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: e092916780716ac80c3608c1bd8ca2901fbb3bd1
Original-Change-Id: Idd3c096a004d76a8fd75df2a884fcb97130d0006
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/262992
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Trybot-Ready: Furquan Shaikh <furquan@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/9898
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
If mmu_init is called more than once then, free_idx should be reset to
1. Here, the assumption would be that mmu_init will not be called more than
once. However, this is not necessarily true. Thus, free_idx should be reset to 1
every time we are initializing ttb from scratch.
BUG=None
BRANCH=None
TEST=Compiles sucessfully and boots to kernel
Change-Id: I5ac0af43346a492583380b0f15101390fc98d182
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 398a68c3b08d82cfa521d235af2c1922629bdf56
Original-Change-Id: Idb7424df7dd577f263f12d1527dbd7fb89216d40
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/216906
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/9068
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
printk() shouldn't be called until the consoles have been
initialized. This just so happened to work by luck. Once
CONFIG_SMP is enabled that breaks because of spinlock
usage in uncached memory.
BUG=chrome-os-partner:31761
BRANCH=None
TEST=Built with CONFIG_SMP and ramstage doesn't hang early.
Change-Id: I54231db3c811c0d19c5c7fbaa406cacd1ff019ec
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 31c3f972ac5c89472009b5b2cb7dbc0f02cfd9a0
Original-Change-Id: I6091b1e949e648b3435231946e5924260bf1807f
Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/216920
Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/9037
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
No need to pass in the same value for the ttb after just
calling mmu_init(). All current users are setting this once
and forgetting it.
BUG=chrome-os-partner:31545
BRANCH=None
TEST=Built and booted on ryu.
Change-Id: Ie446d16eaf4ea65a34a9c76dd7c6c2f9b19c5d57
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: bd77461d483b513a569365673c83badc752f4aa8
Original-Change-Id: I54c7e4892d44ea6129429d8a46461d089dd8e2a9
Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/214772
Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/9016
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
Non-cacheable normal memory is needed when one wants an easy way
to have a DMA region. That way all the reads and writes will be
picked up by the CPU and the device without any cache management
operations.
BUG=chrome-os-partner:31293
BRANCH=None
TEST=With a bevy of other patches can use a carved out DMA region
for talking to USB.
Change-Id: I8172f4b7510dee250aa561d040b27af3080764d7
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: a5bc7ab1709edd97d8795aa9687e6a0edf26ffc6
Original-Change-Id: I36b7fc276467fe3e9cec4d602652d6fa8098c133
Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/212160
Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/8924
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
|
|
Add support for initializing and enabling mmu for armv8. Using 64KiB granule and
33 bits per VA, thus total VA address space is 6GiB. PA Range is 64GiB. Makes
use of memrange library to get a list of all the mmap regions from the SoC to
initialize XLAT table.
Currently, all calculations in mmu.h are based on the assumptions that max 33
bits are used in VA and granule size is 64KiB. Changes in these assumptions will
have to reflect in the dependent calculations as well.
BUG=chrome-os-partner:30688
BRANCH=None
TEST=Compiles rush successfully and boots until "payload not found". Goes past
all the earlier alignment errors.
Original-Change-Id: Iac1df15f0b81dcf64484a56b94f51357bcd67cc2
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/208761
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
(cherry picked from commit 6fe96360c03342115f849074f9e45a2c4e210705)
Signed-off-by: Marc Jones <marc.jones@se-eng.com>
Change-Id: I5360a3be95f198bd0b4f79b62f31228cc7a9c285
Reviewed-on: http://review.coreboot.org/8646
Reviewed-by: Furquan Shaikh <furquan@google.com>
Tested-by: build bot (Jenkins)
|
