A previous change in LibOS started storing fake syscall return
address in R14. This broke stack unwinding when the stack
contained both LibOS syscall and SGX OCALL, because we had no CFI
directives for recovering register values except for RBP, RSP and
RIP, and the value of R14 was lost.
This change adds proper CFI to enclave exit code, by making sure
callee-saved registers can be recovered.
Signed-off-by: Paweł Marczewski <pawel@invisiblethingslab.com>
When an async exception triggers, LibOS needs to know whether it
happened while inside PAL code. Since Linux-PAL sometimes call VDSO
functions, they need to be accounted for as well.
Signed-off-by: Borys Popławski <borysp@invisiblethingslab.com>
These macros were empty and not used anymore. Additionally there was
a bug: `LEAVE_PAL_CALL` actually did not perform `return` and the
execution continued after it.
Signed-off-by: borysp <borysp@invisiblethingslab.com>
Previously, the SGX-signing Python script had a hard-coded value of
1 page. However, the Linux-SGX runtime calculated the SSA frame size
based on the information from CPUID and XFRM. The SSA frame size is
a total of XSAVE area size + GPRs + MISC region, and on feature-rich
CPUs may exceed 1 page. Thus, the SSA frame size in the SIGSTRUCT
(during SGX signing) and in the SECS (during runtime) may mismatch on
such CPUs, and EINIT fails with SGX_INVALID_MEASUREMENT. This commit
simply hard-codes SSA frame size to overapproximated value of 4 pages.
Signed-off-by: Dmitrii Kuvaiskii <dmitrii.kuvaiskii@intel.com>
Change log (most important only):
- unify CPU context structures - now we have only one version -
`PAL_CONTEXT` - which is shared between LibOS and PALs and it should
depend only on the host architecture (not OS),
- syscalls emulation changed:
- dedicated LibOS stack is now used for syscalls emulation,
- removed one indirection level in syscalls table - now it stores
`shim_do_*` functions directly,
- signal handling - completely rewritten:
- all signal queues use proper locking schemes now,
- signals are handled *only* when returning to the user app from LibOS
or PAL,
- nested signals are now possible,
- the app is allowed to jump out of signal handler with the same
sematics as on normal Linux,
- signal altstack is now fully supported,
- syscall restarting is now supported,
- doing a backtrace from the signal handler works properly,
- disallow injecting host-level signals, with one exception, see
`sys.enable_sigterm_injection` manifest option for more details.
These CPUID leaves are used by virtualization software (Hyper-V, KVM,
etc.) and are zeroed out on bare metal. Some runtimes (e.g. JVM) query
them to detect underlying virtualization software. This commit makes
these leaves return zeroes ("no virtualization").
On certain servers the number of cache sets can be greater than
`1 << 16`. This patch increases the sanity check limit to `1 << 30`
to validate against such large number of cache sets.
- Extract parts that are common for all hosts
- Remove some outdated/unnecessary options, we should now be
closer to default configuration
- Disable libthread_db loading (does not work and crashes GDB 9.2)
- Disable pagination when loading debug maps
Intel SGX driver was upstreamed in Linux version 5.11. There, the SGX
device is exposed as `/dev/sgx_enclave` instead of `/dev/sgx/enclave`.
This commit updates link-intel-driver.py to recognize this new name.
This commit also fixes `pseudo_name_ops::list_name()` function pointer
type: `size_t len` argument instead of `int len`. It also adds a
regression test to exercise the newly supported /sys pseudo filesystem.
Replace the old SGX_DBG macro with new subsystem (log_* inside
enclave, urts_log_* outside enclave).
Adjust log levels of some messages, and remove some unnecessary
ones.
The _DkDebugMapAdd function looked for executable program segments
and reported them using ocall_report_mmap(). Unfortunately, the
code incorrectly assumed the ELF header is located at load address
and not start of the map, which is not true for some binaries
(e.g. Ubuntu build of Python), and segfaulted on these binaries.
Because the information about the segments is only important for
SGX profiling, this change simplifies things by removing
the ocall_report_mmap() path and parsing the ELF file in the SGX
subsystem itself.
Instead of 'loader.debug_type', introduce 'loader.log_level'
and 'loader.log_file', along with a set of definitions for
logging at a chosen level.
For now, the call sites keep using the legacy macros (SGX_DBG and
debug()), because converting them all will conflict with other
big changes in the code base. The existing LibOS calls are
assumed to be at 'info' level.
- The GDB command for removing a symbol file takes text address
(or any address inside the mapped memory area), not load offset.
(which might be before the area). Because of that, removing a
map in GDB did not actually work, and displayed a warning.
- The remove_r_debug() function in LibOS did not actually remove
the map from list in LibOS. As a result, LibOS attempted to
report the removal to PAL more than once, causing a harmless but
annoying warning message.
- Use the same mechanism (debug_map) in Pal/Linux and Pal/Linux-SGX.
Previously, Pal/Linux emulated the _r_debug structure, normally
maintained by ld.so, but that cannot be done in SGX outer PAL,
because it's loaded by ld.so already.
- Maintain the debug maps outside of SGX enclave. This allows
initializing them before enclave start, and potentially makes
them easier to use.
- Initialize PAL debug map before enclave start. Previously, this
was done from inside the enclave, so you couldn't set a
breakpoint too early (e.g. in pal_linux_main).
- Store only load address, without list of sections. This is to
avoid parsing the list of sections just to report them to the
debugger. Unfortunately, the GDB version that we support still
needs these sections, but we can retrieve them in GDB plugin.
- Move Python GDB code related to debug maps to a common file.
This commit additionally replaces all `while (true) {}` inf loops with
`die_or_inf_loop` which either crashes the process or loops infinitely
and is not an undefined behavior like the original one (C disallows inf
loops without side effects).
Previously all Linux Pals set `SIGCHILD` signal disposition to `SIG_DFL`
and did not wait for child processes at all. This commit adds zombie
reaping by setting disposition to `SIG_IGN`.
Previously Graphene ORed process start time with host-level pid to
create process id (which was used also as vmid). This could be
problematic as it might cause two process ids to be the same for two
different processes. Why it was actually done remains a mystery ...
This is the next part of the great loader rework, with a lot of breaking changes:
- Complete removal of the "trusted children" thing - now children
processes can be spawned arbitrarily and from arbitrary mountpoint
types, without any additional configuration needed.
- There's a new, required option in the manifest: `libos.entrypoint` - it
specifies the URI to the entry binary in the first process. There's no
need anymore to name the manifest and the first binary identically.
- On SGX, the main binary is not measured in MRENCLAVE anymore - only
PAL, LibOS and the manifest are measured. This is enough to bind
MRENCLAVE to a specific entrypoint user executable if wanted - it
just has to be mounted as a trusted file.
- All Graphene SGX enclaves have now exactly the same MRENCLAVE. This is
a hash of a "Graphene stub", which can "fork" into one of two states
in runtime: initial process or child. The initial process creates a
new "Graphene namespace" with a clean state, it can also be attested
remotely (contrary to child processes). The initial process can spawn
children processes by spawning a Graphene stub and directing it to
start in the child mode. It then attests it locally, and if
successful, establishes an encrypted pipe, "connects" to its own
namespace and treats as trusted (including sending protected files
key).
- Now, there's only one, central manifest describing the initial state
of a Graphene instance which can be spawned from it (previously, each
process required a separate manifest which could have different
configuration - which wasn't actually supported and didn't make sense
design-wise). One downside of central manifests is that all processes
require the same enclave configuration (e.g. size), but that was
already the case so far because of broken checkpointing code. Also,
this is only a temporary problem, which will cease to exist after the
introduction of EDMM.
- `sgx.static_address` was renamed to `sgx.nonpie_binary` and now has to
be inserted manually by users (`sgx_sign` tools doesn't know about the
binaries run inside, which can be even provided or generated in
runtime by the user's workload).
- Caveat: the memory gap for non-PIE executables was removed because it
requires adding a new option to the manifest to be cleanly
implemented. This is left for some future loader rework PR.
GCC (and other compilers, e.g. Clang) provide a stack protector
feature to detect stack corruptions. This is achieved by storing
a 64-bit canary value on the stack frame on function entry and
verifying this value on function exit. Previously, Graphene disabled
stack protector completely. This commit enables it in LibOS and PAL
code (only if `-mstack-protector` feature is supported by compiler).
The stack protector uses a random per-thread canary stored in the
TLS/TCB of each thread. Each PAL implementation must follow the
rule that TLS/TCB is accessed via the GS register and that the offset
of canary in TLS/TCB is 0x8. Since LibOS re-uses TLS/TCB of the PAL,
there is no need for additional enabling at the LibOS layer.
Since `-mstack-protector` feature is architecture-specific, it is
currently enabled only for x86-64 (and above rules on using gs:[0x8]
to access the canary apply only to x86-64).
Co-authored-by: Isaku Yamahata <isaku.yamahata@gmail.com>
Previously, if futex wait returned -EINTR it was returned from
`_DkEventWaitTimeout` too. Now if the waiting condition no longer holds
we ignore EINTR and treat that as a successful wait.
This commit also removes redundant, copy-pasted code (_DkEventWait) and
unused Pal API function (DkEventWait).
This commit adds a new PF utility `pf_tamper` that tampers with
valid protected files and uses this utility to test that the PF
logic in Linux-SGX detects such malicious modifications.
This commit also moves out the PF-format macros and structs from
`protected_files_internal.h` to `protected_files_format.h` for
better readability.
Co-authored-by: Dmitrii Kuvaiskii <dmitrii.kuvaiskii@intel.com>
This functionality is currently not used by Linux-SGX PAL but will be
required in the IOCTL emulation in future commits. Also, function
signatures of ocall_mmap_untrusted() and ocall_munmap_untrusted() now
resemble the signatures of mmap() and munmap().
This function was empty and it is responsibility of the caller (Pal
level) of specific exception handling function (LibOS level) to return
from the exception.
Graphene internal logic as well as application logic relies on most
OCALLs being "uninterruptible", e.g., recv() OCALL is assumed to be
interruptible but mmap() OCALL is assumed to be uninterruptible.
At the same time, Linux-SGX PAL always injects -EINTR in OCALLs when
an async signal arrives from host OS, no matter what OCALL. This
led to spontaneous failures with debug message "ocall returned -4".
This commit adds retrying on all "uninterruptible" OCALLs.
Untrusted Linux-SGX PAL handles host-level asynchronous signals by
emulating the interrupt (-EINTR) of the pending OCALL. Unfortunately,
there was a type cast issue such that int32_t -EINTR (`-4`) was
casted to a positive uint64_t and then OCALL consumed this positive
number instead of erroring out on -EINTR. This commit adds explicit
type casting to fix this bug.
Paweł Marczewski