[Pal/{Linux,Linux-SGX}] Allow IOCTLs and mmaps backed by host devices

- The ioctl() syscalls on device-backed allowed file descriptors are
  pass-through. On the Linux-SGX PAL, the allowed IOCTL requests must
  be explicitly allowed in the manifest via the new option
  `sgx.allowed_ioctls.[id].request`. Also on the Linux-SGX PAL, the
  allowed IOCTLs' arguments (typically pointers to complex nested
  objects) must be explicitly described in the manifest via the new
  options `sgx.ioctl_structs.[id]` and a corresponding reference
  `sgx.allowed_ioctls.[id].struct`; see docs for explanation of
  the IOCTL struct format.
- The mmap() syscalls on device-backed allowed file descriptors are
  pass-through. On the LinuxSGX PAL, all mmapped memory regions reside
  in untrusted memory, and the enclave logic must be careful when
  accessing it (typically the app developer must use this mmapped
  memory region as an encrypted bounce buffer).

Device-backed ioctl() and mmap() syscalls are insecure by themselves
since they only pass the arguments to and from the untrusted memory.
It is the responsibility of the app developer to correctly use them
for secure communication with the host devices.

This commit adds a `device_enclave` LibOS regression test that asks
the Intel SGX driver to create a new enclave and add enclave pages.
This test stresses the new mmap()/ioctl() device-backed syscalls.
This commit is contained in:
Dmitrii Kuvaiskii
2020-11-23 07:54:17 -08:00
parent ca1464c354
commit 37019bd0c9
24 changed files with 1222 additions and 7 deletions
+96
View File
@@ -400,6 +400,102 @@ trusted and allowed are allowed for access, and Graphene-SGX emits a warning
message for every such file. This is a convenient way to determine the set of
files that the ported application uses.
Allowed IOCTLs
^^^^^^^^^^^^^^
::
sgx.ioctl_structs.[identifier] = [memory-layout-format]
sgx.allowed_ioctls.[identifier].request = [NUM]
sgx.allowed_ioctls.[identifier].struct = "[identifier-of-ioctl-struct]"
By default, Graphene-SGX disables all device-backed IOCTLs. This syntax allows
to explicitly allow a set of IOCTLs on devices (devices must be explicitly
mounted via ``fs.mount`` manifest syntax). Only IOCTLs with the ``request``
argument found among the manifest-listed IOCTLs are allowed to pass-through to
the host. Each IOCTL entry must also contain a reference to an IOCTL struct in
its ``struct`` field.
Available IOCTL structs are described via ``sgx.ioctl_structs``. Each IOCTL
struct describes the memory layout of the ``arg`` argument (typically a pointer
to a complex nested object passed to the device). Description of the memory
layout is required for a deep copy of the argument. The memory layout is
described using the TOML syntax of inline arrays (for each new separate memory
region) and inline tables (for each sub-region in one memory region). Each
sub-region is described via the following keys:
- ``name`` is an optional name for this sub-region; mainly used to find
length-specifying fields.
- ``align`` is an optional alignment of the memory region; may be specified only
in the first sub-region of a memory region (all other sub-regions are
contigious with the first sub-region, so specifying their alignment doesn't
make sense).
- ``size`` is a mandatory size of this sub-region. The ``size`` field may be a
string with the name of another field that contains the size value or an
integer with the constant size measured in ``units`` (default unit is 1 byte;
also see below). For example, ``size = "strlen"`` denotes a size field that
will be calculated dynamically during IOCTL execution based on the sub-region
named ``strlen``, whereas ``size = 16`` denotes a sub-region of size 16B. Note
that for ``ptr`` sub-regions, the ``size`` field has a different meaning: it
denotes the number of adjacent memory regions (in other words, it denotes the
number of items in the ``ptr`` array).
- ``unit`` is an optional unit of measurement for ``size``. It is 1 byte by
default. Unit of measurement must be a constant integer. For example,
``size = "strlen"`` and ``unit = 2`` denote a wide-char string (where each
character is 2B long) of a dynamically calculated length.
- ``type = ["none" | "out" | "in" | "inout"]`` is an optional direction of copy
for this sub-region. For example, ``type = "out"`` denotes a sub-region to be
copied out of the enclave to untrusted memory, i.e., this sub-region is an
input to the host device. The default value is ``none`` which is useful for
e.g. padding of structs. This field may be ommitted if the ``ptr`` field is
specified for this sub-region (pointer sub-regions contain the pointer value
which must be rewired to point to untrusted memory).
- ``ptr = [ another memory region ]`` specifies a pointer to another, nested
memory region. This field is required when describing complex IOCTL structs.
Such pointer memory region always has the implicit size of 8B, and the
pointer value is always rewired to the memory region in untrusted memory
(containing a copied-out nested memory region). If ``ptr`` is specified
together with ``size``, it describes not just a pointer but an array of these
memory regions.
- ``onlyif = "simple boolean expression"`` allows to condition the sub region
based on a boolean expression. The only currently supported format of
expressions is ``token1 == token2`` or ``token1 != token2``, where ``token1``
and ``token2`` may be constant integers or sub region names.
Consider this simple example::
sgx.ioctl_structs.st1 = [ { ptr=[ {name="nested_region", align=4096, size=4096, type="out"} ] } ]
The above example specifies a root struct (first memory region) that consists
of a single sub-region that contains an 8-byte pointer value. This pointer
points to another memory region in enclave memory that contains a single
sub-region of size 4KB and that must be 4KB-aligned. This nested sub-region has
a name ``nested_region`` (not used, only for illustrative purposes). Also, this
nested sub-region is copied out of the enclave. The pointer value of the first
memory region is rewired to point to the second memory region in untrusted
memory. No fields/memory regions are copied back from untrusted memory inside
the enclave after an IOCTL with this struct executes.
If the IOCTL's third argument is simply an integer (or unused at all), then the
syntax must specify the struct as an empty TOML array::
sgx.ioctl_structs.st2 = [ ]
IOCTLs that use these structs are defined like this::
sgx.allowed_ioctls.io1.request = 0x12345678
sgx.allowed_ioctls.io1.struct = "st1"
sgx.allowed_ioctls.io2.request = 0x87654321
sgx.allowed_ioctls.io2.struct = "st1"
sgx.allowed_ioctls.io3.request = 0x43218765 # this IOCTL's arg is passed as-is
sgx.allowed_ioctls.io3.struct = "st2"
For more examples and complex usages of the IOCTL syntax, refer to the Graphene
examples, in particular, ``device_enclave.manifest.template``.
Trusted child processes
^^^^^^^^^^^^^^^^^^^^^^^
+5 -2
View File
@@ -931,13 +931,16 @@ static int chroot_checkout(struct shim_handle* hdl) {
if (hdl->fs == &chroot_builtin_fs)
hdl->fs = NULL;
bool is_host_device = false;
if (hdl->type == TYPE_FILE) {
struct shim_file_data* data = FILE_HANDLE_DATA(hdl);
if (data)
if (data) {
is_host_device = (data->type == FILE_DEV && strcmp(qstrgetstr(&hdl->uri), "dev:tty"));
hdl->info.file.data = NULL;
}
}
if (hdl->pal_handle) {
if (hdl->pal_handle && !is_host_device) {
/*
* if the file still exists in the host, no need to send
* the handle over RPC; otherwise, send it.
+40
View File
@@ -6,6 +6,7 @@
*/
#include <asm/ioctls.h>
#include <sys/eventfd.h>
#include "pal.h"
#include "shim_handle.h"
@@ -108,6 +109,45 @@ long shim_do_ioctl(unsigned int fd, unsigned int cmd, unsigned long arg) {
break;
}
if (ret == -ENOSYS && hdl->type == TYPE_FILE && hdl->info.file.type == FILE_DEV) {
/* LibOS doesn't know how to handle this IOCTL, forward it to the host */
ret = 0;
if (!DkDeviceIoControl(hdl->pal_handle, cmd, arg))
ret = -PAL_ERRNO();
/* FIXME: very special case of DRM_IOCTL_I915_GEM_EXECBUFFER2_WR: its arg is of type
* drm_i915_gem_execbuffer2 with a field `rsvd2 >> 32` returning a "fence" FD to poll
* on (basically, an eventfd type of FD): we need to create a corresponding object in
* Graphene, we abuse eventfd for it */
if (cmd == /*DRM_IOCTL_I915_GEM_EXECBUFFER2_WR*/0xc0406469) {
char* arg_char = (char*)arg;
uint64_t rsvd2 = *((uint64_t*)(arg_char + 56)); /* 56 is offset of rsvd2 */
int fence_fd = rsvd2 >> 32;
ret = shim_do_eventfd2(/*count=*/fence_fd, /*flags=*/EFD_SEMAPHORE); /* abuse args */
if (ret >= 0) {
fence_fd = ret;
*((uint64_t*)(arg_char + 56)) = (uint64_t)fence_fd << 32;
}
}
/* FIXME: very special case of DRM_IOCTL_I915_GETPARAM(I915_PARAM_HAS_BSD2): Intel Media
* Driver uses Sys-V IPC (semget and shmget family of syscalls) for multi-process
* user-mode synchronization (to load-balance execution of video encode/decode on
* two VCS rings) if I915_PARAM_HAS_BSD2 == true; we don't support shmget() in
* Graphene so we stub I915_PARAM_HAS_BSD2 = false; this leads to slightly worse
* performance because only one VCS ring is used for video encode/decode but this may
* be fixed after Media Driver removes this Sys-V IPC dependency (see comment
* https://bugzilla.mozilla.org/show_bug.cgi?id=1619585#c46). */
if (cmd == /*DRM_IOCTL_I915_GETPARAM*/0xc0106446) {
typedef struct drm_i915_getparam {int32_t param; int* value;} drm_i915_getparam_t;
drm_i915_getparam_t* arg_getparam = (drm_i915_getparam_t*)arg;
if (arg_getparam->param == /*I915_PARAM_HAS_BSD2*/31) {
/* return BSD2 = false, meaning there is no second VCS ring */
arg_getparam->value = 0;
}
}
}
put_handle(hdl);
return ret;
}
+9
View File
@@ -131,6 +131,15 @@ void* shim_do_mmap(void* addr, size_t length, int prot, int flags, int fd, off_t
flags &= ~MAP_32BIT;
#endif
/* mmap on devices is special: pass-through and not reflected in LibOS's VMA metadata */
if (hdl && hdl->type == TYPE_FILE && hdl->info.file.type == FILE_DEV) {
void* ret_addr = addr;
ret = hdl->fs->fs_ops->mmap(hdl, &ret_addr, length, prot, flags, offset);
if (!ret)
addr = ret_addr;
goto out_handle;
}
if (flags & (MAP_FIXED | MAP_FIXED_NOREPLACE)) {
/* We know that `addr + length` does not overflow (`access_ok` above). */
if (addr < PAL_CB(user_address.start)
+1
View File
@@ -21,6 +21,7 @@
/debug_regs-x86_64
/dev
/device
/device_enclave
/env_from_file
/env_from_host
/epoll_wait_timeout
+6
View File
@@ -16,6 +16,7 @@ c_executables = \
debug \
dev \
device \
device_enclave \
epoll_wait_timeout \
eventfd \
exec \
@@ -94,6 +95,7 @@ manifests = \
attestation.manifest \
debug_log_file.manifest \
debug_log_inline.manifest \
device_enclave.manifest \
env_from_file.manifest \
env_from_host.manifest \
eventfd.manifest \
@@ -122,6 +124,7 @@ exec_target = \
argv_from_file.manifest \
debug_log_file.manifest \
debug_log_inline.manifest \
device_enclave.manifest \
env_from_file.manifest \
env_from_host.manifest \
file_check_policy_allow_all_but_log.manifest \
@@ -177,6 +180,9 @@ CFLAGS-attestation += -I$(PALDIR)/../lib/crypto/mbedtls/crypto/include \
-I$(PALDIR)/../include/pal
LDLIBS-attestation += $(PALDIR)/../lib/crypto/mbedtls/install/lib/libmbedcrypto.a
CFLAGS-device_enclave += -I$(PALDIR)/host/Linux-SGX \
-I$(PALDIR)/../include/pal
CFLAGS-fp_multithread += -pthread -fno-builtin # see comment in the test's source
LDLIBS-fp_multithread += -lm
+164
View File
@@ -0,0 +1,164 @@
#define _GNU_SOURCE
#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include "sgx-driver/sgx.h"
#include "sgx_arch.h"
static int g_isgx_device = -1;
static int read_enclave_token(char* token_name, sgx_arch_token_t* token) {
int token_file = open(token_name, O_RDONLY | O_CLOEXEC, 0);
if (token_file < 0)
err(1, "token file open");
struct stat stat;
int ret = fstat(token_file, &stat);
if (ret < 0)
err(1, "token file fstat");
int bytes = read(token_file, token, sizeof(sgx_arch_token_t));
if (bytes < 0)
err(1, "token file read");
close(token_file);
return 0;
}
static int create_enclave(sgx_arch_secs_t* secs, sgx_arch_token_t* token) {
secs->ssa_frame_size = 2; /* 8192B SSA frame is enough for a dummy enclave */
secs->misc_select = token->masked_misc_select_le;
memcpy(&secs->attributes, &token->body.attributes, sizeof(sgx_attributes_t));
uint64_t request_mmap_addr = secs->base;
uint64_t request_mmap_size = secs->size;
void* addr = mmap((void*)request_mmap_addr, request_mmap_size, PROT_NONE,
MAP_FIXED | MAP_SHARED, g_isgx_device, 0);
if (addr == MAP_FAILED)
err(1, "enclave initial mmap");
struct sgx_enclave_create param = {
.src = (uint64_t)secs,
};
int ret = ioctl(g_isgx_device, SGX_IOC_ENCLAVE_CREATE, &param);
if (ret < 0)
err(1, "ECREATE ioctl");
return 0;
}
static int add_tcs_page_to_enclave(sgx_arch_secs_t* secs) {
int ret;
uint64_t addr = secs->size - 4096;
void* zero_page = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (zero_page == MAP_FAILED)
err(1, "zero-page mmap");
sgx_arch_sec_info_t secinfo;
memset(&secinfo, 0, sizeof(sgx_arch_sec_info_t));
secinfo.flags |= SGX_SECINFO_FLAGS_TCS;
#ifdef SGX_DCAP_16_OR_LATER
/* newer DCAP driver (version 1.6+) allows adding a range of pages for performance, use it */
struct sgx_enclave_add_pages param = {
.offset = addr,
.src = (uint64_t)zero_page,
.length = 4096,
.secinfo = (uint64_t)&secinfo,
.flags = SGX_PAGE_MEASURE,
.count = 0,
};
while (true) {
ret = ioctl(g_isgx_device, SGX_IOC_ENCLAVE_ADD_PAGES, &param);
if (ret < 0) {
if (ret == -EINTR)
continue;
err(1, "ENCLAVE_ADD_PAGES ioctl");
}
break;
}
void* mapped = mmap((void*)(secs->base + addr), 4096, PROT_READ | PROT_WRITE,
MAP_FIXED | MAP_SHARED, g_isgx_device, 0);
if (mapped == MAP_FAILED)
err(1, "ENCLAVE_ADD_PAGES mmap");
#else
/* older drivers (DCAP v1.5- and old out-of-tree) only supports adding one page at a time */
struct sgx_enclave_add_page param = {
.addr = secs->base + addr,
.src = (uint64_t)zero_page,
.secinfo = (uint64_t)&secinfo,
.mrmask = (uint16_t)-1,
};
while (true) {
ret = ioctl(g_isgx_device, SGX_IOC_ENCLAVE_ADD_PAGE, &param);
if (ret < 0) {
if (ret == -EINTR)
continue;
err(1, "ENCLAVE_ADD_PAGE ioctl");
}
break;
}
ret = mprotect((void*)(secs->base + addr), 4096, PROT_READ | PROT_WRITE);
if (ret < 0)
err(1, "ENCLAVE_ADD_PAGE mprotect");
#endif /* SGX_DCAP_16_OR_LATER */
return 0;
}
int main(int argc, char** argv) {
int ret;
sgx_arch_token_t enclave_token;
sgx_arch_secs_t enclave_secs;
g_isgx_device = open(ISGX_FILE, O_RDWR | O_CLOEXEC, 0);
if (g_isgx_device < 0)
err(1, ISGX_FILE " open");
char enclave_token_name[256];
snprintf(enclave_token_name, sizeof(enclave_token_name), "%s.token", argv[0]);
ret = read_enclave_token(enclave_token_name, &enclave_token);
if (ret < 0)
errx(1, "read_enclave_token failed");
memset(&enclave_secs, 0, sizeof(enclave_secs));
enclave_secs.base = 1024UL*1024*1024*1024; /* enclave starts at address 1TB */
enclave_secs.size = 1024UL*1024; /* enclave size is 1MB */
ret = create_enclave(&enclave_secs, &enclave_token);
if (ret < 0)
errx(1, "create_enclave failed");
ret = add_tcs_page_to_enclave(&enclave_secs);
if (ret < 0)
errx(1, "add_tcs_page_to_enclave failed");
ret = ioctl(g_isgx_device, 0x00000001, 123);
if (ret >= 0)
err(1, "unknown ioctl didn't fail");
ret = munmap((void*)enclave_secs.base, enclave_secs.size);
if (ret < 0)
err(1, "destroy enclave failed");
puts("TEST OK");
return 0;
}
@@ -0,0 +1,58 @@
loader.preload = "file:../../src/libsysdb.so"
loader.argv0_override = "device_enclave"
loader.env.LD_LIBRARY_PATH = "/lib:$(ARCH_LIBDIR)"
loader.debug_type = "inline"
fs.mount.graphene_lib.type = "chroot"
fs.mount.graphene_lib.path = "/lib"
fs.mount.graphene_lib.uri = "file:../../../../Runtime"
# new DCAP/in-kernel SGX driver (commented out for Jenkins which uses legacy SGX driver)
fs.mount.devencl.type = "chroot"
fs.mount.devencl.path = "/dev/sgx/enclave"
fs.mount.devencl.uri = "dev:/dev/sgx/enclave"
# legacy SGX driver (used by Jenkins)
#fs.mount.devencl.type = "chroot"
#fs.mount.devencl.path = "/dev/isgx"
#fs.mount.devencl.uri = "dev:/dev/isgx"
sgx.static_address = 1
sgx.trusted_files.ld = "file:../../../../Runtime/ld-linux-x86-64.so.2"
sgx.trusted_files.libc = "file:../../../../Runtime/libc.so.6"
sgx.allowed_files.token = "file:device_enclave.token"
sgx.ioctl_structs.SGX_IOC_ENCLAVE_CREATE = [
{ ptr=[ {name="sgx_arch_secs_t", align=4096, size=4096, type="out"} ] }
]
# IOCTL struct for the new DCAP/in-kernel SGX driver
sgx.ioctl_structs.SGX_IOC_ENCLAVE_ADD_PAGES = [
{ ptr=[ {name="src", align=4096, size="src_length", type="out"} ] },
{ name="offset", size=8, type="out"},
{ name="src_length", size=8, type="out"},
{ ptr=[ {name="secinfo", align=64, size=64, type="out"} ] },
{ name="flags", size=8, type="out"},
{ name="count", size=8, type="in"},
]
# IOCTL struct for the legacy SGX driver
sgx.ioctl_structs.SGX_IOC_ENCLAVE_ADD_PAGE_LEGACY = [
{ name="addr", size=8, type="out"},
{ ptr=[ {name="src", align=4096, size=4096, type="out"} ] },
{ ptr=[ {name="secinfo", align=64, size=64, type="out"} ] },
{ name="mrmask", size=2, type="out"},
]
sgx.allowed_ioctls.SGX_IOC_ENCLAVE_CREATE.request = 0x4008a400
sgx.allowed_ioctls.SGX_IOC_ENCLAVE_CREATE.struct = "SGX_IOC_ENCLAVE_CREATE"
# IOCTL request for the new DCAP/in-kernel SGX driver
sgx.allowed_ioctls.SGX_IOC_ENCLAVE_ADD_PAGES.request = 0xc030a401
sgx.allowed_ioctls.SGX_IOC_ENCLAVE_ADD_PAGES.struct = "SGX_IOC_ENCLAVE_ADD_PAGES"
# IOCTL request for the legacy SGX driver
sgx.allowed_ioctls.SGX_IOC_ENCLAVE_ADD_PAGE_LEGACY.request = 0x401aa401
sgx.allowed_ioctls.SGX_IOC_ENCLAVE_ADD_PAGE_LEGACY.struct = "SGX_IOC_ENCLAVE_ADD_PAGE_LEGACY"
+6
View File
@@ -592,6 +592,12 @@ class TC_40_FileSystem(RegressionTestCase):
stdout, _ = self.run_binary(['device'])
self.assertIn('TEST OK', stdout)
@unittest.skipUnless(HAS_SGX,
'This test uses the Intel SGX driver (host-Linux kernel module).')
def test_003_device_enclave(self):
stdout, _ = self.run_binary(['device_enclave'])
self.assertIn('TEST OK', stdout)
def test_010_path(self):
stdout, _ = self.run_binary(['proc_path'])
self.assertIn('proc path test success', stdout)
+17
View File
@@ -737,6 +737,23 @@ PAL_PTR DkSegmentRegister(PAL_FLG reg, PAL_PTR addr);
*/
PAL_NUM DkMemoryAvailableQuota(void);
/*!
* \brief Perform a device-specific operation `cmd`.
*
* This function corresponds to ioctl() in UNIX systems and DeviceIoControl() in Windows.
*
* \param[in] handle Handle of the device.
* \param[in] cmd Device-dependent request/control code.
* \param[in,out] arg Arbitrary argument to `cmd`. May be unused or used as a 64-bit integer
* or used as a pointer to a buffer that contains the data required to
* perform the operation as well as the data returned by the operation. For
* some PALs (e.g., Linux-SGX PAL), the manifest must describe the layout of
* this buffer in order to correctly copy the data to/from the host.
*
* \return Returns 1 on success, 0 on failure. Use PAL_ERRNO() to get the actual error code.
*/
PAL_BOL DkDeviceIoControl(PAL_HANDLE handle, PAL_NUM cmd, PAL_NUM arg);
/*!
* \brief Obtain the attestation report (local) with `user_report_data` embedded into it.
*
+11
View File
@@ -99,6 +99,17 @@ DkCpuIdRetrieve(PAL_IDX leaf, PAL_IDX subleaf, PAL_IDX values[4]) {
LEAVE_PAL_CALL_RETURN(PAL_TRUE);
}
PAL_BOL DkDeviceIoControl(PAL_HANDLE handle, PAL_NUM cmd, PAL_NUM arg) {
ENTER_PAL_CALL(DkDeviceIoControl);
int ret = _DkDeviceIoControl(handle, cmd, arg);
if (ret < 0) {
_DkRaiseFailure(-ret);
LEAVE_PAL_CALL_RETURN(PAL_FALSE);
}
LEAVE_PAL_CALL_RETURN(PAL_TRUE);
}
PAL_BOL DkAttestationReport(PAL_PTR user_report_data, PAL_NUM* user_report_data_size,
PAL_PTR target_info, PAL_NUM* target_info_size, PAL_PTR report,
PAL_NUM* report_size) {
+1 -1
View File
@@ -143,7 +143,7 @@ int _DkStreamOpen(PAL_HANDLE* handle, const char* uri, int access, int share, in
assert(WITHIN_MASK(access, PAL_ACCESS_MASK));
assert(WITHIN_MASK(share, PAL_SHARE_MASK));
assert(WITHIN_MASK(create, PAL_CREATE_MASK));
// assert(WITHIN_MASK(create, PAL_CREATE_MASK)); /* FIXME: eventfd abuses create arg */
assert(WITHIN_MASK(options, PAL_OPTION_MASK));
int ret = parse_stream_uri(&uri, &type, &ops);
+702
View File
@@ -17,6 +17,7 @@
#include "pal_internal.h"
#include "pal_linux.h"
#include "pal_linux_error.h"
#include "toml.h"
static int dev_open(PAL_HANDLE* handle, const char* type, const char* uri, int access, int share,
int create, int options) {
@@ -111,6 +112,19 @@ static int64_t dev_write(PAL_HANDLE handle, uint64_t offset, uint64_t size, cons
return IS_ERR(bytes) ? unix_to_pal_error(ERRNO(bytes)) : bytes;
}
static int dev_map(PAL_HANDLE handle, void** addr, int prot, uint64_t offset, uint64_t size) {
if (!IS_HANDLE_TYPE(handle, dev))
return -PAL_ERROR_INVAL;
if (handle->dev.fd == PAL_IDX_POISON)
return -PAL_ERROR_DENIED;
assert(WITHIN_MASK(prot, PAL_PROT_MASK));
int ret = ocall_mmap_untrusted(handle->dev.fd, offset, size, PAL_PROT_TO_LINUX(prot), addr);
return IS_ERR(ret) ? unix_to_pal_error(ERRNO(ret)) : ret;
}
static int dev_close(PAL_HANDLE handle) {
if (!IS_HANDLE_TYPE(handle, dev))
return -PAL_ERROR_INVAL;
@@ -210,8 +224,696 @@ struct handle_ops g_dev_ops = {
.open = &dev_open,
.read = &dev_read,
.write = &dev_write,
.map = &dev_map,
.close = &dev_close,
.flush = &dev_flush,
.attrquery = &dev_attrquery,
.attrquerybyhdl = &dev_attrquerybyhdl,
};
/*
* Code below describes the deep-copy syntax in the TOML manifest used for copying complex nested
* objects out and in the SGX enclave. This syntax is currently used for IOCTL emulation. This
* syntax is generic enough to describe any memory layout for deep copy.
*
* The following example describes the main implementation details:
*
* struct pascal_str { uint8_t len; char str[]; };
* struct c_str { char str[]; };
* struct root { struct pascal_str* s1; struct c_str* s2; uint64_t s2_len; int8_t x; int8_t y; };
*
* alignas(128) struct root obj;
* ioctl(devfd, _IOWR(DEVICE_MAGIC, DEVICE_FUNC, struct root), &obj);
*
* The example IOCTL takes as a third argument a pointer to an object of type `struct root` that
* contains two pointers to other objects (pascal-style string and a C-style string) and embeds two
* integers `x` and `y`. The two strings reside in separate memory regions in enclave memory. Note
* that the length of the C-style string is stored in the `s2_len` field of the root object. The
* `pascal_str` string is an input to the IOCTL, the `c_str` string is both input and output of the
* IOCTL, and the integers `x` and `y` are both outputs of the IOCTL. Also note that the root
* object is 128B-aligned (for illustration purposes).
*
* The corresponding deep-copy syntax in TOML looks like this:
*
* sgx.ioctl_structs.ROOT_FOR_DEVICE_FUNC = [
* { align = 128, ptr = [ {name="pascal-str-len", size=1, type="out"},
* {name="pascal-str", size="pascal-str-len", type="out"} ] },
* { ptr = [ {name="c-str", size="c-str-len", type="inout"} ], size = 1 },
* { name = "c-str-len", size = 8, unit = 1, type = "in" },
* { onlyif = "c-str-len == pascal-str-len", size = 2, type = "in" }
* { onlyif = "c-str-len != pascal-str-len", size = 2, type = "none" }
* ]
*
* One can observe the following rules in this TOML syntax:
* 1. Each separate memory region is represented as a TOML array (`[]`).
* 2. Each sub-region of one memory region is represented as a TOML table (`{}`).
* 3. Each sub-region may be a pointer (`ptr`) to another memory region. In this case, the value of
* `ptr` is a TOML-array representation of that other memory region. The `ptr` sub-region always
* has size of 8B (assuming x86-64) and doesn't have an in/out type. The `size` field of the
* `ptr` sub-region has a different meaning than for non-pointer sub-regions: it is the number
* of adjacent memory regions that this pointer points to (i.e. it describes an array).
* 4. Sub-regions can be fixed-size (like the last sub-region containing two bytes `x` and `y`) or
* can be flexible-size (like the two strings). In the latter case, the `size` field contains a
* name of a sub-region where the actual size is stored.
* 5. Sub-regions that store the size of another sub-region must be 1, 2, 4, or 8 bytes in size.
* 6. Sub-regions may have a name for ease of identification; this is required for "size"
* sub-regions but may be omitted for all other kinds of sub-regions.
* 7. Sub-regions may have one of the four types: "out" to copy contents of the sub-region outside
* the enclave to untrusted memory, "in" to copy from untrusted memory to inside the enclave,
* "inout" to copy in both directions, "none" to not copy at all (useful for e.g. padding).
* Note that pointer sub-regions do not have a type.
* 8. The first sub-region (and only the first!) may specify the alignment of the memory region.
* 9. The total size of a sub-region is calculated as `size * unit`. By default `unit` is 1 byte.
* 10. Sub-regions may be conditioned using `onlyif = "simple boolean expression"`. In the example
* above, `x` and `y` will be copied back into the enclave only if the two strings have the
* same length; otherwise `x` and `y` will not be modified. The only currently supported format
* of expressions is "token1 == token2" or "token1 != token2", where `token1` and `token2` may
* be constant integers or sub region names.
*
* The diagram below shows how this complex object is copied from enclave memory (left side) to
* untrusted memory (right side). MR stands for "memory region", SR stands for "sub-region". Note
* how enclave pointers are copied and rewired to point to untrusted memory regions.
*
* struct root (MR1) | deep-copied struct (aligned at 128B)
* +------------------+ | +------------------------+
* +----+ pascal_str* s1 | SR1 | +----+ pascal_str* s1 (MR1)|
* | | | | | | |
* | | c_str* s2 +-------+ SR2 | | | c_str* s2 +-------------+
* | | | | | | | | |
* | | uint64_t s2_len | | SR3 | | | uint64_t s2_len | |
* | | | | | | | | |
* | | int8_t x, y | | SR4 | | | int8_t x=0, y=0 | |
* | +------------------+ | | | +------------------------+ |
* | | | +->| uint8_t len (MR2)| |
* v (MR2) | | | | |
* +-------------+ | | | char str[] | |
* | uint8_t len | | SR5 | +------------------------+ |
* | | | | | char str[] (MR3)|<-+
* | char str[] | | SR6 | +------------------------+
* +-------------+ | |
* (MR3) v |
* +----------+-+ |
* | char str[] | SR7 |
* +------------+ |
*
*/
/* for simplicity and thread-safety, we allocate mem_regions and sub_regions on stack; we assume
* that deep copy of objects doesn't exceed the specified limits of memory and sub regions */
#define MAX_MEM_REGIONS 32
#define MAX_SUB_REGIONS 64
/* direction of copy: none (used for padding), out of enclave, inside enclave, both or a special
* "pointer" sub-region; default is COPY_NONE_ENCLAVE */
enum mem_copy_type {COPY_NONE_ENCLAVE = 0, COPY_OUT_ENCLAVE, COPY_IN_ENCLAVE, COPY_INOUT_ENCLAVE,
COPY_PTR_ENCLAVE};
struct mem_region {
toml_array_t* toml_array; /* describes contigious sub_regions in this mem_region */
void* encl_addr; /* base address of this memory region in enclave memory */
bool adjacent; /* memory region adjacent to previous one? (used for arrays) */
};
struct sub_region {
enum mem_copy_type type; /* direction of copy during OCALL (or pointer to another region) */
char* name; /* may be NULL for unnamed regions */
ssize_t align; /* alignment of this sub-region */
ssize_t size; /* may be dynamically determined from another sub-region */
char* size_name; /* needed if "size" sub region is defined after this sub region */
ssize_t unit; /* total size in bytes is calculated as `size * unit` */
void* encl_addr; /* base address of this sub region in enclave memory */
void* untrusted_addr; /* base address of the corresponding sub region in untrusted memory */
toml_array_t* mem_ptr; /* for pointers/arrays, specifies pointed-to mem region */
};
/* finds a sub region with name `sub_region_name` among `sub_regions` and reads the value in it */
static int get_sub_region_value(struct sub_region* sub_regions, int sub_regions_cnt,
const char* sub_region_name, ssize_t* value) {
/* it is important to iterate in reverse order because there may be an array of mem regions
* with same-named sub regions, and we want to find the "latest size" sub region, i.e. the one
* belonging to the same mem region */
for (int i = sub_regions_cnt - 1; i >= 0; i--) {
if (sub_regions[i].name && !strcmp(sub_regions[i].name, sub_region_name)) {
/* found corresponding sub region, read its value */
if (!sub_regions[i].encl_addr || sub_regions[i].encl_addr == (void*)-1) {
/* enclave address is invalid, user provided bad struct */
return -EFAULT;
}
if (sub_regions[i].size == sizeof(uint8_t)) {
*value = (ssize_t)(*((uint8_t*)sub_regions[i].encl_addr));
} else if (sub_regions[i].size == sizeof(uint16_t)) {
*value = (ssize_t)(*((uint16_t*)sub_regions[i].encl_addr));
} else if (sub_regions[i].size == sizeof(uint32_t)) {
*value = (ssize_t)(*((uint32_t*)sub_regions[i].encl_addr));
} else if (sub_regions[i].size == sizeof(uint64_t)) {
*value = (ssize_t)(*((uint64_t*)sub_regions[i].encl_addr));
} else {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (deep-copy sub-entry \'%s\' must be of "
"legitimate size: 1, 2, 4 or 8 bytes)\n",
sub_regions[i].name);
return -EINVAL;
}
return 0;
}
}
return -ENOENT;
}
/* Parses a simple boolean expression in `expr`, finds corresponding values from sub regions and
* calculates the resulting boolean value. NOTE: currently supports only "x == y" and "x != y"
* where both "x" and "y" can be sub region names or constant integers. */
static int calculate_boolean_expr(struct sub_region* sub_regions, int sub_regions_cnt, char* expr,
bool* value) {
int ret;
char* cur = expr;
while (*cur == ' ' || *cur == '\t') { cur++; }
/* read first token */
char* token1 = cur;
while (isalnum(*cur) || *cur == '_' || *cur == '-') { cur++; }
size_t token1_len = cur - token1;
while (*cur == ' ' || *cur == '\t') { cur++; }
/* read comparator */
char* compar = cur;
while (*cur == '=' || *cur == '!') { cur++; }
size_t compar_len = cur - compar;
while (*cur == ' ' || *cur == '\t') { cur++; }
/* read second token */
char* token2 = cur;
while (isalnum(*cur) || *cur == '_' || *cur == '-') { cur++; }
size_t token2_len = cur - token2;
while (*cur == ' ' || *cur == '\t') { cur++; }
/* make sure the whole string is in format "token1 == token2" or "token1 != token2" */
if (compar_len != 2 || (memcmp(compar, "==", 2) && (memcmp(compar, "!=", 2)))) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (the only supported comparators are \'==\' and "
"\'!=\' but they are not found in expression \'%s\')\n", expr);
return -EINVAL;
}
if (*cur != '\0' || !token1_len || !token2_len) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (cannot parse expression \'%s\')\n", expr);
return -EINVAL;
}
/* get actual values for two tokens */
char* endptr = NULL;
token1[token1_len] = '\0';
ssize_t val1 = strtol(token1, &endptr, /*base=*/0);
if (endptr != token1 + token1_len) {
/* could not read the constant integer, the token must be a string-name of a sub region */
ret = get_sub_region_value(sub_regions, sub_regions_cnt, token1, &val1);
if (ret < 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (cannot find first sub region in expression "
"\'%s\')\n", expr);
return ret;
}
}
token2[token2_len] = '\0';
ssize_t val2 = strtol(token2, &endptr, /*base=*/0);
if (endptr != token2 + token2_len) {
/* could not read the constant integer, the token must be a string-name of a sub region */
ret = get_sub_region_value(sub_regions, sub_regions_cnt, token2, &val2);
if (ret < 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (cannot find second sub region in expression "
"\'%s\')\n", expr);
return ret;
}
}
if (!(memcmp(compar, "==", 2))) {
*value = val1 == val2;
} else {
*value = val1 != val2;
}
return 0;
}
/* caller sets `_sub_regions_cnt` to maximum number of sub_regions; this variable is updated to
* return the number of actually used sub_regions */
static int collect_sub_regions(toml_array_t* root_toml_array, void* root_encl_addr,
struct sub_region* sub_regions, int* _sub_regions_cnt) {
int ret;
assert(root_toml_array && toml_array_nelem(root_toml_array) > 0);
assert(sub_regions && _sub_regions_cnt);
int max_sub_regions = *_sub_regions_cnt;
int sub_regions_cnt = 0;
for (int i = 0; i < max_sub_regions; i++) {
sub_regions[i].align = 0;
sub_regions[i].unit = 0;
sub_regions[i].size = -1;
sub_regions[i].name = NULL;
sub_regions[i].size_name = NULL;
sub_regions[i].encl_addr = NULL;
sub_regions[i].untrusted_addr = NULL;
sub_regions[i].mem_ptr = NULL;
}
struct mem_region mem_regions[MAX_MEM_REGIONS] = {0};
mem_regions[0].toml_array = root_toml_array;
mem_regions[0].encl_addr = root_encl_addr;
mem_regions[0].adjacent = false;
int mem_regions_cnt = 1;
/* collecting memory regions and their sub-regions must use breadth-first search to dynamically
* calculate sizes of sub-regions even if they are specified via another sub-region's "name" */
char* cur_encl_addr = NULL; /* char* type for pointer arithmetic */
int mem_region_idx = 0;
while (mem_region_idx < mem_regions_cnt) {
struct mem_region* cur_mem_region = &mem_regions[mem_region_idx];
mem_region_idx++;
if (!cur_mem_region->adjacent)
cur_encl_addr = cur_mem_region->encl_addr;
int cur_mem_region_first_sub_region = sub_regions_cnt;
for (int i = 0; i < toml_array_nelem(cur_mem_region->toml_array); i++) {
toml_table_t* sub_region_info = toml_table_at(cur_mem_region->toml_array, i);
if (!sub_region_info) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (each memory subregion must be a TOML "
"table)\n");
ret = -EINVAL;
goto out;
}
if (sub_regions_cnt == max_sub_regions) {
SGX_DBG(DBG_E, "Too many memory sub-regions in a deep-copy syntax (maximum "
"possible is %d)\n", max_sub_regions);
ret = -ENOMEM;
goto out;
}
struct sub_region* cur_sub_region = &sub_regions[sub_regions_cnt];
sub_regions_cnt++;
cur_sub_region->encl_addr = cur_encl_addr;
if (!cur_encl_addr || cur_encl_addr == (void*)-1) {
/* enclave address is invalid, user provided bad struct */
ret = -EFAULT;
goto out;
}
toml_raw_t sub_region_onlyif_raw = toml_raw_in(sub_region_info, "onlyif");
toml_raw_t sub_region_name_raw = toml_raw_in(sub_region_info, "name");
toml_raw_t sub_region_type_raw = toml_raw_in(sub_region_info, "type");
toml_raw_t sub_region_align_raw = toml_raw_in(sub_region_info, "align");
toml_raw_t sub_region_size_raw = toml_raw_in(sub_region_info, "size");
toml_raw_t sub_region_unit_raw = toml_raw_in(sub_region_info, "unit");
toml_array_t* sub_region_ptr_arr = toml_array_in(sub_region_info, "ptr");
if (sub_region_align_raw && i != 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'align\' may be specified only for the "
"first sub-region of the memory region)\n");
ret = -EINVAL;
goto out;
}
if (sub_region_type_raw && sub_region_ptr_arr) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'ptr\' sub-entries cannot specify "
"a \'type\'; pointers are never copied directly but rewired)\n");
ret = -EINVAL;
goto out;
}
if (sub_region_onlyif_raw) {
char* onlyif_expr = NULL;
ret = toml_rtos(sub_region_onlyif_raw, &onlyif_expr);
if (ret < 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'onlyif\' of a deep-copy sub-entry "
"must be a TOML string surrounded by double quotes)\n");
ret = -EINVAL;
goto out;
}
bool val = false;
/* "sub_regions_cnt - 1" is to exclude myself */
ret = calculate_boolean_expr(sub_regions, sub_regions_cnt - 1, onlyif_expr, &val);
if (ret < 0) {
free(onlyif_expr);
goto out;
}
free(onlyif_expr);
if (!val) {
/* onlyif expression is false, we must skip this sub region completely */
sub_regions_cnt--;
continue;
}
}
cur_sub_region->name = NULL;
if (sub_region_name_raw) {
ret = toml_rtos(sub_region_name_raw, &cur_sub_region->name);
if (ret < 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'name\' of a deep-copy sub-entry "
"must be a TOML string surrounded by double quotes)\n");
ret = -EINVAL;
goto out;
}
}
cur_sub_region->type = COPY_NONE_ENCLAVE;
if (sub_region_type_raw) {
char* type_str = NULL;
ret = toml_rtos(sub_region_type_raw, &type_str);
if (ret < 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'type\' of a deep-copy sub-entry "
"must be a TOML string surrounded by double quotes)\n");
ret = -EINVAL;
goto out;
}
if (!strcmp(type_str, "out")) {
cur_sub_region->type = COPY_OUT_ENCLAVE;
}
else if (!strcmp(type_str, "in")) {
cur_sub_region->type = COPY_IN_ENCLAVE;
}
else if (!strcmp(type_str, "inout")) {
cur_sub_region->type = COPY_INOUT_ENCLAVE;
}
else if (!strcmp(type_str, "none")) {
cur_sub_region->type = COPY_NONE_ENCLAVE;
}
else {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'type\' of a deep-copy sub-entry "
"must be one of \"out\", \"in\", \"inout\" or \"none\")\n");
free(type_str);
ret = -EINVAL;
goto out;
}
free(type_str);
}
cur_sub_region->align = 0;
if (sub_region_align_raw) {
ret = toml_rtoi(sub_region_align_raw, &cur_sub_region->align);
if (ret < 0 || cur_sub_region->align <= 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'align\' of a deep-copy sub-entry "
"must be a positive number)\n");
ret = -EINVAL;
goto out;
}
}
if (sub_region_ptr_arr) {
/* only set type for now, we postpone pointer/array handling for later */
cur_sub_region->type = COPY_PTR_ENCLAVE;
cur_sub_region->mem_ptr = sub_region_ptr_arr;
}
cur_sub_region->size = -1;
if (sub_region_size_raw) {
ret = toml_rtos(sub_region_size_raw, &cur_sub_region->size_name);
if (ret == 0) {
ssize_t val = -1;
/* "sub_regions_cnt - 1" is to exclude myself; do not fail if couldn't find
* (we will try later one more time) */
ret = get_sub_region_value(sub_regions, sub_regions_cnt - 1,
cur_sub_region->size_name, &val);
if (ret < 0 && ret != -ENOENT) {
goto out;
}
cur_sub_region->size = val;
} else {
/* size is specified not as string (another sub-region's name), then must be
* specified explicitly as number of bytes */
ret = toml_rtoi(sub_region_size_raw, &cur_sub_region->size);
if (ret < 0 || cur_sub_region->size <= 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'size\' of a deep-copy "
"sub-entry must be a TOML string or a positive number)\n");
ret = -EINVAL;
goto out;
}
}
}
cur_sub_region->unit = 1; /* 1 byte by default */
if (sub_region_unit_raw) {
ret = toml_rtoi(sub_region_unit_raw, &cur_sub_region->unit);
if (ret < 0 || cur_sub_region->unit <= 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'unit\' of a deep-copy sub-entry "
"must be a positive number)\n");
ret = -EINVAL;
goto out;
}
}
cur_sub_region->size *= cur_sub_region->unit;
cur_encl_addr += cur_sub_region->type == COPY_PTR_ENCLAVE ? 8 : cur_sub_region->size;
}
/* iterate through collected pointer/array sub regions and add corresponding mem regions */
for (int i = cur_mem_region_first_sub_region; i < sub_regions_cnt; i++) {
if (sub_regions[i].type != COPY_PTR_ENCLAVE)
continue;
if (sub_regions[i].size >= 0) {
/* sizes was found in the first swoop, nothing to do here */
} else if (sub_regions[i].size < 0 && sub_regions[i].size_name) {
/* pointer/array size was not found in the first swoop, try again */
ssize_t val = -1;
ret = get_sub_region_value(sub_regions, sub_regions_cnt, sub_regions[i].size_name,
&val);
if (ret < 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (cannot find sub region \'%s\')\n",
sub_regions[i].size_name);
goto out;
}
sub_regions[i].size = val;
} else {
/* size is not specified at all for this sub region, assume it is 1 */
sub_regions[i].size = 1;
}
for (ssize_t k = 0; k < sub_regions[i].size; k++) {
if (mem_regions_cnt == MAX_MEM_REGIONS) {
SGX_DBG(DBG_E, "Too many memory regions in a deep-copy syntax (maximum "
"possible is %d)\n", MAX_MEM_REGIONS);
ret = -ENOMEM;
goto out;
}
void* mem_region_addr = *((void**)sub_regions[i].encl_addr);
if (!mem_region_addr)
continue;
mem_regions[mem_regions_cnt].toml_array = sub_regions[i].mem_ptr;
mem_regions[mem_regions_cnt].encl_addr = mem_region_addr;
mem_regions[mem_regions_cnt].adjacent = k > 0;
mem_regions_cnt++;
}
sub_regions[i].size = sizeof(void*); /* rewire to actual size of sub-region */
}
}
*_sub_regions_cnt = sub_regions_cnt;
ret = 0;
out:
#if 0
for (int i = 0; i < sub_regions_cnt; i++) {
/* we print 4B memory cells in hope that this memory cell is accessible (only for debug) */
uint32_t value = sub_regions[i].encl_addr ? *((uint32_t*)sub_regions[i].encl_addr) : 0;
SGX_DBG(DBG_E, "===== sub_region[%d]: \'%s\', size=%ld, addr=%p, value=%u\n",
i, sub_regions[i].name, sub_regions[i].size, sub_regions[i].encl_addr, value);
}
#endif
for (int i = 0; i < sub_regions_cnt; i++) {
if (ret == 0) {
/* misc sanity checks on all collected sub regions */
if (sub_regions[i].size < 0) {
SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'size\' is unspecified/negative in sub "
"region \'%s\')\n", sub_regions[i].name);
ret = -EINVAL;
}
if (sub_regions[i].size > 0 && (!sub_regions[i].encl_addr ||
sub_regions[i].encl_addr == (void*)-1)) {
/* enclave address is invalid, user provided bad struct */
ret = -EFAULT;
}
}
/* "name" fields are not needed after we collected all sub_regions */
free(sub_regions[i].name);
free(sub_regions[i].size_name);
sub_regions[i].name = NULL;
sub_regions[i].size_name = NULL;
}
return ret;
}
static void copy_sub_regions_to_untrusted(struct sub_region* sub_regions, int sub_regions_cnt,
void* untrusted_addr) {
char* cur_untrusted_addr = untrusted_addr; /* char* type for pointer arithmetic */
for (int i = 0; i < sub_regions_cnt; i++) {
if (sub_regions[i].align > 0)
cur_untrusted_addr = ALIGN_UP_PTR(cur_untrusted_addr, sub_regions[i].align);
sub_regions[i].untrusted_addr = cur_untrusted_addr;
if (sub_regions[i].type == COPY_OUT_ENCLAVE || sub_regions[i].type == COPY_INOUT_ENCLAVE)
memcpy(cur_untrusted_addr, sub_regions[i].encl_addr, sub_regions[i].size);
cur_untrusted_addr += sub_regions[i].size;
}
for (int i = 0; i < sub_regions_cnt; i++) {
if (sub_regions[i].type == COPY_PTR_ENCLAVE) {
void* encl_ptr_value = *((void**)sub_regions[i].encl_addr);
/* rewire pointer value in untrusted memory to a corresponding untrusted sub-region */
for (int j = 0; j < sub_regions_cnt; j++) {
if (sub_regions[j].encl_addr == encl_ptr_value) {
*((void**)sub_regions[i].untrusted_addr) = sub_regions[j].untrusted_addr;
break;
}
}
}
}
}
static void copy_sub_regions_to_enclave(struct sub_region* sub_regions, int sub_regions_cnt) {
for (int i = 0; i < sub_regions_cnt; i++) {
if (sub_regions[i].type == COPY_IN_ENCLAVE || sub_regions[i].type == COPY_INOUT_ENCLAVE)
memcpy(sub_regions[i].encl_addr, sub_regions[i].untrusted_addr, sub_regions[i].size);
}
}
int _DkDeviceIoControl(PAL_HANDLE handle, unsigned int cmd, uint64_t arg) {
int ret;
if (!IS_HANDLE_TYPE(handle, dev))
return -PAL_ERROR_INVAL;
if (handle->dev.fd == PAL_IDX_POISON)
return -PAL_ERROR_DENIED;
toml_table_t* manifest_sgx = toml_table_in(g_pal_state.manifest_root, "sgx");
if (!manifest_sgx)
return -PAL_ERROR_NOTIMPLEMENTED;
toml_table_t* toml_allowed_ioctls = toml_table_in(manifest_sgx, "allowed_ioctls");
if (!toml_allowed_ioctls)
return -PAL_ERROR_NOTIMPLEMENTED;
ssize_t toml_allowed_ioctls_cnt = toml_table_ntab(toml_allowed_ioctls);
if (toml_allowed_ioctls_cnt <= 0)
return -PAL_ERROR_NOTIMPLEMENTED;
for (ssize_t i = 0; i < toml_allowed_ioctls_cnt; i++) {
const char* toml_allowed_ioctl_key = toml_key_in(toml_allowed_ioctls, i);
assert(toml_allowed_ioctl_key);
toml_table_t* toml_ioctl_table = toml_table_in(toml_allowed_ioctls, toml_allowed_ioctl_key);
if (!toml_ioctl_table)
continue;
toml_raw_t toml_ioctl_request_raw = toml_raw_in(toml_ioctl_table, "request");
if (!toml_ioctl_request_raw)
continue;
int64_t ioctl_request = 0x0;
ret = toml_rtoi(toml_ioctl_request_raw, &ioctl_request);
if (ret < 0 || ioctl_request == 0x0) {
SGX_DBG(DBG_E, "Invalid request value of allowed ioctl \'%s\' in manifest\n",
toml_allowed_ioctl_key);
continue;
}
if (ioctl_request == (int64_t)cmd) {
/* found this IOCTL request in the manifest, now must find the corresponding struct */
toml_raw_t toml_ioctl_struct_raw = toml_raw_in(toml_ioctl_table, "struct");
if (!toml_ioctl_struct_raw) {
SGX_DBG(DBG_E, "Cannot find struct value of allowed ioctl \'%s\' in manifest\n",
toml_allowed_ioctl_key);
return -PAL_ERROR_NOTIMPLEMENTED;
}
char* ioctl_struct_str = NULL;
ret = toml_rtos(toml_ioctl_struct_raw, &ioctl_struct_str);
if (ret < 0) {
SGX_DBG(DBG_E, "Invalid struct value of allowed ioctl \'%s\' in manifest "
"(sgx.allowed_ioctls.[identifier].struct must be a TOML string)\n",
toml_allowed_ioctl_key);
return -PAL_ERROR_INVAL;
}
toml_table_t* toml_ioctl_structs = toml_table_in(manifest_sgx, "ioctl_structs");
if (!toml_ioctl_structs) {
SGX_DBG(DBG_E, "There are no ioctl structs found in manifest\n");
free(ioctl_struct_str);
return -PAL_ERROR_INVAL;
}
toml_array_t* toml_ioctl_struct = toml_array_in(toml_ioctl_structs, ioctl_struct_str);
if (!toml_ioctl_struct) {
SGX_DBG(DBG_E, "Cannot find struct value \'%s\' of allowed ioctl \'%s\' in "
"manifest (or it is not a correctly formatted TOML array)\n",
ioctl_struct_str, toml_allowed_ioctl_key);
free(ioctl_struct_str);
return -PAL_ERROR_INVAL;
}
free(ioctl_struct_str);
if (toml_array_nelem(toml_ioctl_struct) == 0) {
/* special case of an empty TOML array == base-type or ignored IOCTL argument */
ret = ocall_ioctl(handle->dev.fd, cmd, arg);
return ret < 0 ? unix_to_pal_error(ERRNO(ret)) : 0;
}
/* typical case of used IOCTL argument: deep-copy the IOCTL argument's input data
* outside of enclave, execute the IOCTL OCALL, and deep-copy the IOCTL argument's
* output data back into enclave */
struct sub_region sub_regions[MAX_SUB_REGIONS];
int sub_regions_cnt = MAX_SUB_REGIONS;
ret = collect_sub_regions(toml_ioctl_struct, (void*)arg, sub_regions, &sub_regions_cnt);
if (ret < 0) {
if (ret != -EFAULT) {
SGX_DBG(DBG_E, "Invalid struct format of allowed ioctl \'%s\' in manifest\n",
toml_allowed_ioctl_key);
}
return unix_to_pal_error(ERRNO(ret));
}
void* untrusted_addr = NULL;
size_t untrusted_size = 0;
for (int i = 0; i < sub_regions_cnt; i++) {
assert(sub_regions[i].size > 0);
untrusted_size += sub_regions[i].size + sub_regions[i].align;
}
ret = ocall_mmap_untrusted(/*fd=*/-1, /*offset=*/0, untrusted_size,
PROT_READ | PROT_WRITE, &untrusted_addr);
if (ret < 0) {
return -PAL_ERROR_NOMEM;
}
copy_sub_regions_to_untrusted(sub_regions, sub_regions_cnt, untrusted_addr);
ret = ocall_ioctl(handle->dev.fd, cmd, (uint64_t)untrusted_addr);
if (ret < 0) {
ocall_munmap_untrusted(untrusted_addr, untrusted_size);
return unix_to_pal_error(ERRNO(ret));
}
copy_sub_regions_to_enclave(sub_regions, sub_regions_cnt);
ocall_munmap_untrusted(untrusted_addr, untrusted_size);
return 0;
}
}
return -PAL_ERROR_NOTIMPLEMENTED;
}
+8 -2
View File
@@ -48,8 +48,14 @@ static int eventfd_pal_open(PAL_HANDLE* handle, const char* type, const char* ur
return -PAL_ERROR_INVAL;
}
/* Using create arg as a work-around (note: initval is uint32 but create is int32).*/
ret = ocall_eventfd(create, eventfd_type(options));
if (options & PAL_OPTION_EFD_SEMAPHORE) {
/* FIXME: semaphore option is abused to hint that `count` already contains host FD;
* currently used for LibOS emulation of DRM_IOCTL_I915_GEM_EXECBUFFER2_WR ioctl */
ret = create;
} else {
/* Using create arg as a work-around (note: initval is uint32 but create is int32).*/
ret = ocall_eventfd(create, eventfd_type(options));
}
if (IS_ERR(ret))
return unix_to_pal_error(ERRNO(ret));
+21
View File
@@ -1580,3 +1580,24 @@ int ocall_sched_getaffinity(void* tcs, size_t cpumask_size, void* cpu_mask) {
sgx_reset_ustack(old_ustack);
return retval;
}
int ocall_ioctl(int fd, unsigned int cmd, unsigned long arg) {
int retval = 0;
ms_ocall_ioctl_t* ms;
void* old_ustack = sgx_prepare_ustack();
ms = sgx_alloc_on_ustack_aligned(sizeof(*ms), alignof(*ms));
if (!ms) {
sgx_reset_ustack(old_ustack);
return -EPERM;
}
WRITE_ONCE(ms->ms_fd, fd);
WRITE_ONCE(ms->ms_cmd, cmd);
WRITE_ONCE(ms->ms_arg, arg);
retval = sgx_exitless_ocall(OCALL_IOCTL, ms);
sgx_reset_ustack(old_ustack);
return retval;
}
+2
View File
@@ -96,6 +96,8 @@ int ocall_update_debugger(struct debug_map* _Atomic* debug_map);
int ocall_eventfd(unsigned int initval, int flags);
int ocall_ioctl(int fd, unsigned int cmd, unsigned long arg);
/*!
* \brief Execute untrusted code in PAL to obtain a quote from the Quoting Enclave.
*
+7
View File
@@ -62,6 +62,7 @@ enum {
OCALL_DELETE,
OCALL_UPDATE_DEBUGGER,
OCALL_EVENTFD,
OCALL_IOCTL,
OCALL_GET_QUOTE,
OCALL_NR,
};
@@ -290,6 +291,12 @@ typedef struct {
int ms_flags;
} ms_ocall_eventfd_t;
typedef struct {
int ms_fd;
unsigned int ms_cmd;
unsigned long ms_arg;
} ms_ocall_ioctl_t;
typedef struct {
bool ms_is_epid;
sgx_spid_t ms_spid;
+8
View File
@@ -682,6 +682,13 @@ static long sgx_ocall_eventfd(void* pms) {
return ret;
}
static long sgx_ocall_ioctl(void* pms) {
ms_ocall_ioctl_t* ms = (ms_ocall_ioctl_t*)pms;
ODEBUG(OCALL_IOCTL, ms);
long ret = INLINE_SYSCALL(ioctl, 3, ms->ms_fd, ms->ms_cmd, ms->ms_arg);
return ret;
}
static long sgx_ocall_update_debugger(void* pms) {
ms_ocall_update_debugger_t* ms = (ms_ocall_update_debugger_t*)pms;
ODEBUG(OCALL_UPDATE_DEBUGGER, ms);
@@ -742,6 +749,7 @@ sgx_ocall_fn_t ocall_table[OCALL_NR] = {
[OCALL_DELETE] = sgx_ocall_delete,
[OCALL_UPDATE_DEBUGGER] = sgx_ocall_update_debugger,
[OCALL_EVENTFD] = sgx_ocall_eventfd,
[OCALL_IOCTL] = sgx_ocall_ioctl,
[OCALL_GET_QUOTE] = sgx_ocall_get_quote,
};
+32
View File
@@ -110,6 +110,26 @@ static int64_t dev_write(PAL_HANDLE handle, uint64_t offset, uint64_t size, cons
return IS_ERR(bytes) ? unix_to_pal_error(ERRNO(bytes)) : bytes;
}
static int dev_map(PAL_HANDLE handle, void** addr, int prot, uint64_t offset, uint64_t size) {
if (!IS_HANDLE_TYPE(handle, dev))
return -PAL_ERROR_INVAL;
if (handle->dev.fd == PAL_IDX_POISON)
return -PAL_ERROR_DENIED;
assert(WITHIN_MASK(prot, PAL_PROT_MASK));
void* mem = *addr;
int flags = MAP_FILE | PAL_MEM_FLAGS_TO_LINUX(0, prot) | (mem ? MAP_FIXED : 0);
mem = (void*)ARCH_MMAP(mem, size, PAL_PROT_TO_LINUX(prot), flags, handle->dev.fd, offset);
if (IS_ERR_P(mem))
return unix_to_pal_error(ERRNO_P(mem));
*addr = mem;
return 0;
}
static int dev_close(PAL_HANDLE handle) {
if (!IS_HANDLE_TYPE(handle, dev))
return -PAL_ERROR_INVAL;
@@ -201,8 +221,20 @@ struct handle_ops g_dev_ops = {
.open = &dev_open,
.read = &dev_read,
.write = &dev_write,
.map = &dev_map,
.close = &dev_close,
.flush = &dev_flush,
.attrquery = &dev_attrquery,
.attrquerybyhdl = &dev_attrquerybyhdl,
};
int _DkDeviceIoControl(PAL_HANDLE handle, unsigned int cmd, uint64_t arg) {
if (!IS_HANDLE_TYPE(handle, dev))
return -PAL_ERROR_INVAL;
if (handle->dev.fd == PAL_IDX_POISON)
return -PAL_ERROR_DENIED;
int ret = INLINE_SYSCALL(ioctl, 3, handle->dev.fd, cmd, arg);
return IS_ERR(ret) ? unix_to_pal_error(ERRNO(ret)) : 0;
}
+8 -2
View File
@@ -50,8 +50,14 @@ static int eventfd_pal_open(PAL_HANDLE* handle, const char* type, const char* ur
return -PAL_ERROR_INVAL;
}
/* Using create arg as a work-around (note: initval is uint32 but create is int32).*/
ret = INLINE_SYSCALL(eventfd2, 2, create, eventfd_type(options));
if (options & PAL_OPTION_EFD_SEMAPHORE) {
/* FIXME: semaphore option is abused to hint that `count` already contains host FD;
* currently used for LibOS emulation of DRM_IOCTL_I915_GEM_EXECBUFFER2_WR ioctl */
ret = create;
} else {
/* Using create arg as a work-around (note: initval is uint32 but create is int32).*/
ret = INLINE_SYSCALL(eventfd2, 2, create, eventfd_type(options));
}
if (IS_ERR(ret))
return unix_to_pal_error(ERRNO(ret));
+12
View File
@@ -25,6 +25,10 @@ static int64_t dev_write(PAL_HANDLE handle, uint64_t offset, uint64_t size, cons
return -PAL_ERROR_NOTIMPLEMENTED;
}
static int dev_map(PAL_HANDLE handle, void** addr, int prot, uint64_t offset, uint64_t size) {
return -PAL_ERROR_NOTIMPLEMENTED;
}
static int dev_close(PAL_HANDLE handle) {
return -PAL_ERROR_NOTIMPLEMENTED;
}
@@ -45,8 +49,16 @@ struct handle_ops g_dev_ops = {
.open = &dev_open,
.read = &dev_read,
.write = &dev_write,
.map = &dev_map,
.close = &dev_close,
.flush = &dev_flush,
.attrquery = &dev_attrquery,
.attrquerybyhdl = &dev_attrquerybyhdl,
};
int _DkDeviceIoControl(PAL_HANDLE handle, unsigned int cmd, uint64_t arg) {
__UNUSED(handle);
__UNUSED(cmd);
__UNUSED(arg);
return -PAL_ERROR_NOTIMPLEMENTED;
}
+1
View File
@@ -45,6 +45,7 @@ DkStreamAttributesSetByHandle
DkMemoryAvailableQuota
DkDebugAttachBinary
DkDebugDetachBinary
DkDeviceIoControl
DkAttestationReport
DkAttestationQuote
DkSetProtectedFilesKey
+1
View File
@@ -282,6 +282,7 @@ int _DkSegmentRegisterSet(int reg, const void* addr);
int _DkSegmentRegisterGet(int reg, void** addr);
int _DkInstructionCacheFlush(const void* addr, int size);
int _DkCpuIdRetrieve(unsigned int leaf, unsigned int subleaf, unsigned int values[4]);
int _DkDeviceIoControl(PAL_HANDLE handle, unsigned int cmd, uint64_t arg);
int _DkAttestationReport(PAL_PTR user_report_data, PAL_NUM* user_report_data_size,
PAL_PTR target_info, PAL_NUM* target_info_size, PAL_PTR report,
PAL_NUM* report_size);
+6
View File
@@ -71,6 +71,12 @@ def read_manifest(filename):
manifest_layout.append((None, None))
break
if line.strip().startswith("{") or line.strip().startswith("}") or \
line.strip().startswith("[") or line.strip().startswith("]"):
# TOML multi-line syntax, add it as-is
manifest_layout.append((None, line))
continue
pound = line.find("#")
if pound != -1:
comment = line[pound:].strip()