diff --git a/Documentation/manifest-syntax.rst b/Documentation/manifest-syntax.rst index 2588caa3..0db05d88 100644 --- a/Documentation/manifest-syntax.rst +++ b/Documentation/manifest-syntax.rst @@ -400,6 +400,102 @@ trusted and allowed are allowed for access, and Graphene-SGX emits a warning message for every such file. This is a convenient way to determine the set of files that the ported application uses. +Allowed IOCTLs +^^^^^^^^^^^^^^ + +:: + + sgx.ioctl_structs.[identifier] = [memory-layout-format] + + sgx.allowed_ioctls.[identifier].request = [NUM] + sgx.allowed_ioctls.[identifier].struct = "[identifier-of-ioctl-struct]" + +By default, Graphene-SGX disables all device-backed IOCTLs. This syntax allows +to explicitly allow a set of IOCTLs on devices (devices must be explicitly +mounted via ``fs.mount`` manifest syntax). Only IOCTLs with the ``request`` +argument found among the manifest-listed IOCTLs are allowed to pass-through to +the host. Each IOCTL entry must also contain a reference to an IOCTL struct in +its ``struct`` field. + +Available IOCTL structs are described via ``sgx.ioctl_structs``. Each IOCTL +struct describes the memory layout of the ``arg`` argument (typically a pointer +to a complex nested object passed to the device). Description of the memory +layout is required for a deep copy of the argument. The memory layout is +described using the TOML syntax of inline arrays (for each new separate memory +region) and inline tables (for each sub-region in one memory region). Each +sub-region is described via the following keys: + +- ``name`` is an optional name for this sub-region; mainly used to find + length-specifying fields. +- ``align`` is an optional alignment of the memory region; may be specified only + in the first sub-region of a memory region (all other sub-regions are + contigious with the first sub-region, so specifying their alignment doesn't + make sense). +- ``size`` is a mandatory size of this sub-region. The ``size`` field may be a + string with the name of another field that contains the size value or an + integer with the constant size measured in ``units`` (default unit is 1 byte; + also see below). For example, ``size = "strlen"`` denotes a size field that + will be calculated dynamically during IOCTL execution based on the sub-region + named ``strlen``, whereas ``size = 16`` denotes a sub-region of size 16B. Note + that for ``ptr`` sub-regions, the ``size`` field has a different meaning: it + denotes the number of adjacent memory regions (in other words, it denotes the + number of items in the ``ptr`` array). +- ``unit`` is an optional unit of measurement for ``size``. It is 1 byte by + default. Unit of measurement must be a constant integer. For example, + ``size = "strlen"`` and ``unit = 2`` denote a wide-char string (where each + character is 2B long) of a dynamically calculated length. +- ``type = ["none" | "out" | "in" | "inout"]`` is an optional direction of copy + for this sub-region. For example, ``type = "out"`` denotes a sub-region to be + copied out of the enclave to untrusted memory, i.e., this sub-region is an + input to the host device. The default value is ``none`` which is useful for + e.g. padding of structs. This field may be ommitted if the ``ptr`` field is + specified for this sub-region (pointer sub-regions contain the pointer value + which must be rewired to point to untrusted memory). +- ``ptr = [ another memory region ]`` specifies a pointer to another, nested + memory region. This field is required when describing complex IOCTL structs. + Such pointer memory region always has the implicit size of 8B, and the + pointer value is always rewired to the memory region in untrusted memory + (containing a copied-out nested memory region). If ``ptr`` is specified + together with ``size``, it describes not just a pointer but an array of these + memory regions. +- ``onlyif = "simple boolean expression"`` allows to condition the sub region + based on a boolean expression. The only currently supported format of + expressions is ``token1 == token2`` or ``token1 != token2``, where ``token1`` + and ``token2`` may be constant integers or sub region names. + +Consider this simple example:: + + sgx.ioctl_structs.st1 = [ { ptr=[ {name="nested_region", align=4096, size=4096, type="out"} ] } ] + +The above example specifies a root struct (first memory region) that consists +of a single sub-region that contains an 8-byte pointer value. This pointer +points to another memory region in enclave memory that contains a single +sub-region of size 4KB and that must be 4KB-aligned. This nested sub-region has +a name ``nested_region`` (not used, only for illustrative purposes). Also, this +nested sub-region is copied out of the enclave. The pointer value of the first +memory region is rewired to point to the second memory region in untrusted +memory. No fields/memory regions are copied back from untrusted memory inside +the enclave after an IOCTL with this struct executes. + +If the IOCTL's third argument is simply an integer (or unused at all), then the +syntax must specify the struct as an empty TOML array:: + + sgx.ioctl_structs.st2 = [ ] + +IOCTLs that use these structs are defined like this:: + + sgx.allowed_ioctls.io1.request = 0x12345678 + sgx.allowed_ioctls.io1.struct = "st1" + + sgx.allowed_ioctls.io2.request = 0x87654321 + sgx.allowed_ioctls.io2.struct = "st1" + + sgx.allowed_ioctls.io3.request = 0x43218765 # this IOCTL's arg is passed as-is + sgx.allowed_ioctls.io3.struct = "st2" + +For more examples and complex usages of the IOCTL syntax, refer to the Graphene +examples, in particular, ``device_enclave.manifest.template``. + Trusted child processes ^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/LibOS/shim/src/fs/chroot/fs.c b/LibOS/shim/src/fs/chroot/fs.c index ae8d3a04..4ef8ccfd 100644 --- a/LibOS/shim/src/fs/chroot/fs.c +++ b/LibOS/shim/src/fs/chroot/fs.c @@ -931,13 +931,16 @@ static int chroot_checkout(struct shim_handle* hdl) { if (hdl->fs == &chroot_builtin_fs) hdl->fs = NULL; + bool is_host_device = false; if (hdl->type == TYPE_FILE) { struct shim_file_data* data = FILE_HANDLE_DATA(hdl); - if (data) + if (data) { + is_host_device = (data->type == FILE_DEV && strcmp(qstrgetstr(&hdl->uri), "dev:tty")); hdl->info.file.data = NULL; + } } - if (hdl->pal_handle) { + if (hdl->pal_handle && !is_host_device) { /* * if the file still exists in the host, no need to send * the handle over RPC; otherwise, send it. diff --git a/LibOS/shim/src/sys/shim_ioctl.c b/LibOS/shim/src/sys/shim_ioctl.c index 5fe4ed2c..05495118 100644 --- a/LibOS/shim/src/sys/shim_ioctl.c +++ b/LibOS/shim/src/sys/shim_ioctl.c @@ -6,6 +6,7 @@ */ #include +#include #include "pal.h" #include "shim_handle.h" @@ -108,6 +109,45 @@ long shim_do_ioctl(unsigned int fd, unsigned int cmd, unsigned long arg) { break; } + if (ret == -ENOSYS && hdl->type == TYPE_FILE && hdl->info.file.type == FILE_DEV) { + /* LibOS doesn't know how to handle this IOCTL, forward it to the host */ + ret = 0; + if (!DkDeviceIoControl(hdl->pal_handle, cmd, arg)) + ret = -PAL_ERRNO(); + + /* FIXME: very special case of DRM_IOCTL_I915_GEM_EXECBUFFER2_WR: its arg is of type + * drm_i915_gem_execbuffer2 with a field `rsvd2 >> 32` returning a "fence" FD to poll + * on (basically, an eventfd type of FD): we need to create a corresponding object in + * Graphene, we abuse eventfd for it */ + if (cmd == /*DRM_IOCTL_I915_GEM_EXECBUFFER2_WR*/0xc0406469) { + char* arg_char = (char*)arg; + uint64_t rsvd2 = *((uint64_t*)(arg_char + 56)); /* 56 is offset of rsvd2 */ + int fence_fd = rsvd2 >> 32; + ret = shim_do_eventfd2(/*count=*/fence_fd, /*flags=*/EFD_SEMAPHORE); /* abuse args */ + if (ret >= 0) { + fence_fd = ret; + *((uint64_t*)(arg_char + 56)) = (uint64_t)fence_fd << 32; + } + } + + /* FIXME: very special case of DRM_IOCTL_I915_GETPARAM(I915_PARAM_HAS_BSD2): Intel Media + * Driver uses Sys-V IPC (semget and shmget family of syscalls) for multi-process + * user-mode synchronization (to load-balance execution of video encode/decode on + * two VCS rings) if I915_PARAM_HAS_BSD2 == true; we don't support shmget() in + * Graphene so we stub I915_PARAM_HAS_BSD2 = false; this leads to slightly worse + * performance because only one VCS ring is used for video encode/decode but this may + * be fixed after Media Driver removes this Sys-V IPC dependency (see comment + * https://bugzilla.mozilla.org/show_bug.cgi?id=1619585#c46). */ + if (cmd == /*DRM_IOCTL_I915_GETPARAM*/0xc0106446) { + typedef struct drm_i915_getparam {int32_t param; int* value;} drm_i915_getparam_t; + drm_i915_getparam_t* arg_getparam = (drm_i915_getparam_t*)arg; + if (arg_getparam->param == /*I915_PARAM_HAS_BSD2*/31) { + /* return BSD2 = false, meaning there is no second VCS ring */ + arg_getparam->value = 0; + } + } + } + put_handle(hdl); return ret; } diff --git a/LibOS/shim/src/sys/shim_mmap.c b/LibOS/shim/src/sys/shim_mmap.c index 4e2e5403..b68a4f8e 100644 --- a/LibOS/shim/src/sys/shim_mmap.c +++ b/LibOS/shim/src/sys/shim_mmap.c @@ -131,6 +131,15 @@ void* shim_do_mmap(void* addr, size_t length, int prot, int flags, int fd, off_t flags &= ~MAP_32BIT; #endif + /* mmap on devices is special: pass-through and not reflected in LibOS's VMA metadata */ + if (hdl && hdl->type == TYPE_FILE && hdl->info.file.type == FILE_DEV) { + void* ret_addr = addr; + ret = hdl->fs->fs_ops->mmap(hdl, &ret_addr, length, prot, flags, offset); + if (!ret) + addr = ret_addr; + goto out_handle; + } + if (flags & (MAP_FIXED | MAP_FIXED_NOREPLACE)) { /* We know that `addr + length` does not overflow (`access_ok` above). */ if (addr < PAL_CB(user_address.start) diff --git a/LibOS/shim/test/regression/.gitignore b/LibOS/shim/test/regression/.gitignore index c6f5bd96..275e585a 100644 --- a/LibOS/shim/test/regression/.gitignore +++ b/LibOS/shim/test/regression/.gitignore @@ -21,6 +21,7 @@ /debug_regs-x86_64 /dev /device +/device_enclave /env_from_file /env_from_host /epoll_wait_timeout diff --git a/LibOS/shim/test/regression/Makefile b/LibOS/shim/test/regression/Makefile index d74204b6..8b4914ed 100644 --- a/LibOS/shim/test/regression/Makefile +++ b/LibOS/shim/test/regression/Makefile @@ -16,6 +16,7 @@ c_executables = \ debug \ dev \ device \ + device_enclave \ epoll_wait_timeout \ eventfd \ exec \ @@ -94,6 +95,7 @@ manifests = \ attestation.manifest \ debug_log_file.manifest \ debug_log_inline.manifest \ + device_enclave.manifest \ env_from_file.manifest \ env_from_host.manifest \ eventfd.manifest \ @@ -122,6 +124,7 @@ exec_target = \ argv_from_file.manifest \ debug_log_file.manifest \ debug_log_inline.manifest \ + device_enclave.manifest \ env_from_file.manifest \ env_from_host.manifest \ file_check_policy_allow_all_but_log.manifest \ @@ -177,6 +180,9 @@ CFLAGS-attestation += -I$(PALDIR)/../lib/crypto/mbedtls/crypto/include \ -I$(PALDIR)/../include/pal LDLIBS-attestation += $(PALDIR)/../lib/crypto/mbedtls/install/lib/libmbedcrypto.a +CFLAGS-device_enclave += -I$(PALDIR)/host/Linux-SGX \ + -I$(PALDIR)/../include/pal + CFLAGS-fp_multithread += -pthread -fno-builtin # see comment in the test's source LDLIBS-fp_multithread += -lm diff --git a/LibOS/shim/test/regression/device_enclave.c b/LibOS/shim/test/regression/device_enclave.c new file mode 100644 index 00000000..8efa4bad --- /dev/null +++ b/LibOS/shim/test/regression/device_enclave.c @@ -0,0 +1,164 @@ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "sgx-driver/sgx.h" +#include "sgx_arch.h" + +static int g_isgx_device = -1; + +static int read_enclave_token(char* token_name, sgx_arch_token_t* token) { + int token_file = open(token_name, O_RDONLY | O_CLOEXEC, 0); + if (token_file < 0) + err(1, "token file open"); + + struct stat stat; + int ret = fstat(token_file, &stat); + if (ret < 0) + err(1, "token file fstat"); + + int bytes = read(token_file, token, sizeof(sgx_arch_token_t)); + if (bytes < 0) + err(1, "token file read"); + + close(token_file); + return 0; +} + +static int create_enclave(sgx_arch_secs_t* secs, sgx_arch_token_t* token) { + secs->ssa_frame_size = 2; /* 8192B SSA frame is enough for a dummy enclave */ + secs->misc_select = token->masked_misc_select_le; + memcpy(&secs->attributes, &token->body.attributes, sizeof(sgx_attributes_t)); + + uint64_t request_mmap_addr = secs->base; + uint64_t request_mmap_size = secs->size; + + void* addr = mmap((void*)request_mmap_addr, request_mmap_size, PROT_NONE, + MAP_FIXED | MAP_SHARED, g_isgx_device, 0); + + if (addr == MAP_FAILED) + err(1, "enclave initial mmap"); + + struct sgx_enclave_create param = { + .src = (uint64_t)secs, + }; + + int ret = ioctl(g_isgx_device, SGX_IOC_ENCLAVE_CREATE, ¶m); + if (ret < 0) + err(1, "ECREATE ioctl"); + + return 0; +} + +static int add_tcs_page_to_enclave(sgx_arch_secs_t* secs) { + int ret; + uint64_t addr = secs->size - 4096; + + void* zero_page = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (zero_page == MAP_FAILED) + err(1, "zero-page mmap"); + + sgx_arch_sec_info_t secinfo; + memset(&secinfo, 0, sizeof(sgx_arch_sec_info_t)); + secinfo.flags |= SGX_SECINFO_FLAGS_TCS; + +#ifdef SGX_DCAP_16_OR_LATER + /* newer DCAP driver (version 1.6+) allows adding a range of pages for performance, use it */ + struct sgx_enclave_add_pages param = { + .offset = addr, + .src = (uint64_t)zero_page, + .length = 4096, + .secinfo = (uint64_t)&secinfo, + .flags = SGX_PAGE_MEASURE, + .count = 0, + }; + + while (true) { + ret = ioctl(g_isgx_device, SGX_IOC_ENCLAVE_ADD_PAGES, ¶m); + if (ret < 0) { + if (ret == -EINTR) + continue; + err(1, "ENCLAVE_ADD_PAGES ioctl"); + } + break; + } + + void* mapped = mmap((void*)(secs->base + addr), 4096, PROT_READ | PROT_WRITE, + MAP_FIXED | MAP_SHARED, g_isgx_device, 0); + if (mapped == MAP_FAILED) + err(1, "ENCLAVE_ADD_PAGES mmap"); +#else + /* older drivers (DCAP v1.5- and old out-of-tree) only supports adding one page at a time */ + struct sgx_enclave_add_page param = { + .addr = secs->base + addr, + .src = (uint64_t)zero_page, + .secinfo = (uint64_t)&secinfo, + .mrmask = (uint16_t)-1, + }; + + while (true) { + ret = ioctl(g_isgx_device, SGX_IOC_ENCLAVE_ADD_PAGE, ¶m); + if (ret < 0) { + if (ret == -EINTR) + continue; + err(1, "ENCLAVE_ADD_PAGE ioctl"); + } + break; + } + + ret = mprotect((void*)(secs->base + addr), 4096, PROT_READ | PROT_WRITE); + if (ret < 0) + err(1, "ENCLAVE_ADD_PAGE mprotect"); +#endif /* SGX_DCAP_16_OR_LATER */ + + return 0; +} + +int main(int argc, char** argv) { + int ret; + sgx_arch_token_t enclave_token; + sgx_arch_secs_t enclave_secs; + + g_isgx_device = open(ISGX_FILE, O_RDWR | O_CLOEXEC, 0); + if (g_isgx_device < 0) + err(1, ISGX_FILE " open"); + + char enclave_token_name[256]; + snprintf(enclave_token_name, sizeof(enclave_token_name), "%s.token", argv[0]); + ret = read_enclave_token(enclave_token_name, &enclave_token); + if (ret < 0) + errx(1, "read_enclave_token failed"); + + memset(&enclave_secs, 0, sizeof(enclave_secs)); + enclave_secs.base = 1024UL*1024*1024*1024; /* enclave starts at address 1TB */ + enclave_secs.size = 1024UL*1024; /* enclave size is 1MB */ + ret = create_enclave(&enclave_secs, &enclave_token); + if (ret < 0) + errx(1, "create_enclave failed"); + + ret = add_tcs_page_to_enclave(&enclave_secs); + if (ret < 0) + errx(1, "add_tcs_page_to_enclave failed"); + + ret = ioctl(g_isgx_device, 0x00000001, 123); + if (ret >= 0) + err(1, "unknown ioctl didn't fail"); + + ret = munmap((void*)enclave_secs.base, enclave_secs.size); + if (ret < 0) + err(1, "destroy enclave failed"); + + puts("TEST OK"); + return 0; +} diff --git a/LibOS/shim/test/regression/device_enclave.manifest.template b/LibOS/shim/test/regression/device_enclave.manifest.template new file mode 100644 index 00000000..ad35856c --- /dev/null +++ b/LibOS/shim/test/regression/device_enclave.manifest.template @@ -0,0 +1,58 @@ +loader.preload = "file:../../src/libsysdb.so" +loader.argv0_override = "device_enclave" +loader.env.LD_LIBRARY_PATH = "/lib:$(ARCH_LIBDIR)" +loader.debug_type = "inline" + +fs.mount.graphene_lib.type = "chroot" +fs.mount.graphene_lib.path = "/lib" +fs.mount.graphene_lib.uri = "file:../../../../Runtime" + +# new DCAP/in-kernel SGX driver (commented out for Jenkins which uses legacy SGX driver) +fs.mount.devencl.type = "chroot" +fs.mount.devencl.path = "/dev/sgx/enclave" +fs.mount.devencl.uri = "dev:/dev/sgx/enclave" + +# legacy SGX driver (used by Jenkins) +#fs.mount.devencl.type = "chroot" +#fs.mount.devencl.path = "/dev/isgx" +#fs.mount.devencl.uri = "dev:/dev/isgx" + +sgx.static_address = 1 + +sgx.trusted_files.ld = "file:../../../../Runtime/ld-linux-x86-64.so.2" +sgx.trusted_files.libc = "file:../../../../Runtime/libc.so.6" + +sgx.allowed_files.token = "file:device_enclave.token" + +sgx.ioctl_structs.SGX_IOC_ENCLAVE_CREATE = [ + { ptr=[ {name="sgx_arch_secs_t", align=4096, size=4096, type="out"} ] } +] + +# IOCTL struct for the new DCAP/in-kernel SGX driver +sgx.ioctl_structs.SGX_IOC_ENCLAVE_ADD_PAGES = [ + { ptr=[ {name="src", align=4096, size="src_length", type="out"} ] }, + { name="offset", size=8, type="out"}, + { name="src_length", size=8, type="out"}, + { ptr=[ {name="secinfo", align=64, size=64, type="out"} ] }, + { name="flags", size=8, type="out"}, + { name="count", size=8, type="in"}, +] + +# IOCTL struct for the legacy SGX driver +sgx.ioctl_structs.SGX_IOC_ENCLAVE_ADD_PAGE_LEGACY = [ + { name="addr", size=8, type="out"}, + { ptr=[ {name="src", align=4096, size=4096, type="out"} ] }, + { ptr=[ {name="secinfo", align=64, size=64, type="out"} ] }, + { name="mrmask", size=2, type="out"}, +] + +sgx.allowed_ioctls.SGX_IOC_ENCLAVE_CREATE.request = 0x4008a400 +sgx.allowed_ioctls.SGX_IOC_ENCLAVE_CREATE.struct = "SGX_IOC_ENCLAVE_CREATE" + +# IOCTL request for the new DCAP/in-kernel SGX driver +sgx.allowed_ioctls.SGX_IOC_ENCLAVE_ADD_PAGES.request = 0xc030a401 +sgx.allowed_ioctls.SGX_IOC_ENCLAVE_ADD_PAGES.struct = "SGX_IOC_ENCLAVE_ADD_PAGES" + +# IOCTL request for the legacy SGX driver +sgx.allowed_ioctls.SGX_IOC_ENCLAVE_ADD_PAGE_LEGACY.request = 0x401aa401 +sgx.allowed_ioctls.SGX_IOC_ENCLAVE_ADD_PAGE_LEGACY.struct = "SGX_IOC_ENCLAVE_ADD_PAGE_LEGACY" diff --git a/LibOS/shim/test/regression/test_libos.py b/LibOS/shim/test/regression/test_libos.py index 0061630d..1bbb0e32 100644 --- a/LibOS/shim/test/regression/test_libos.py +++ b/LibOS/shim/test/regression/test_libos.py @@ -592,6 +592,12 @@ class TC_40_FileSystem(RegressionTestCase): stdout, _ = self.run_binary(['device']) self.assertIn('TEST OK', stdout) + @unittest.skipUnless(HAS_SGX, + 'This test uses the Intel SGX driver (host-Linux kernel module).') + def test_003_device_enclave(self): + stdout, _ = self.run_binary(['device_enclave']) + self.assertIn('TEST OK', stdout) + def test_010_path(self): stdout, _ = self.run_binary(['proc_path']) self.assertIn('proc path test success', stdout) diff --git a/Pal/include/pal/pal.h b/Pal/include/pal/pal.h index 07f9c586..d6b2ef9d 100644 --- a/Pal/include/pal/pal.h +++ b/Pal/include/pal/pal.h @@ -737,6 +737,23 @@ PAL_PTR DkSegmentRegister(PAL_FLG reg, PAL_PTR addr); */ PAL_NUM DkMemoryAvailableQuota(void); +/*! + * \brief Perform a device-specific operation `cmd`. + * + * This function corresponds to ioctl() in UNIX systems and DeviceIoControl() in Windows. + * + * \param[in] handle Handle of the device. + * \param[in] cmd Device-dependent request/control code. + * \param[in,out] arg Arbitrary argument to `cmd`. May be unused or used as a 64-bit integer + * or used as a pointer to a buffer that contains the data required to + * perform the operation as well as the data returned by the operation. For + * some PALs (e.g., Linux-SGX PAL), the manifest must describe the layout of + * this buffer in order to correctly copy the data to/from the host. + * + * \return Returns 1 on success, 0 on failure. Use PAL_ERRNO() to get the actual error code. + */ +PAL_BOL DkDeviceIoControl(PAL_HANDLE handle, PAL_NUM cmd, PAL_NUM arg); + /*! * \brief Obtain the attestation report (local) with `user_report_data` embedded into it. * diff --git a/Pal/src/db_misc.c b/Pal/src/db_misc.c index 9d0d3483..f3e280f3 100644 --- a/Pal/src/db_misc.c +++ b/Pal/src/db_misc.c @@ -99,6 +99,17 @@ DkCpuIdRetrieve(PAL_IDX leaf, PAL_IDX subleaf, PAL_IDX values[4]) { LEAVE_PAL_CALL_RETURN(PAL_TRUE); } +PAL_BOL DkDeviceIoControl(PAL_HANDLE handle, PAL_NUM cmd, PAL_NUM arg) { + ENTER_PAL_CALL(DkDeviceIoControl); + + int ret = _DkDeviceIoControl(handle, cmd, arg); + if (ret < 0) { + _DkRaiseFailure(-ret); + LEAVE_PAL_CALL_RETURN(PAL_FALSE); + } + LEAVE_PAL_CALL_RETURN(PAL_TRUE); +} + PAL_BOL DkAttestationReport(PAL_PTR user_report_data, PAL_NUM* user_report_data_size, PAL_PTR target_info, PAL_NUM* target_info_size, PAL_PTR report, PAL_NUM* report_size) { diff --git a/Pal/src/db_streams.c b/Pal/src/db_streams.c index 5318ea8e..35cacef9 100644 --- a/Pal/src/db_streams.c +++ b/Pal/src/db_streams.c @@ -143,7 +143,7 @@ int _DkStreamOpen(PAL_HANDLE* handle, const char* uri, int access, int share, in assert(WITHIN_MASK(access, PAL_ACCESS_MASK)); assert(WITHIN_MASK(share, PAL_SHARE_MASK)); - assert(WITHIN_MASK(create, PAL_CREATE_MASK)); +// assert(WITHIN_MASK(create, PAL_CREATE_MASK)); /* FIXME: eventfd abuses create arg */ assert(WITHIN_MASK(options, PAL_OPTION_MASK)); int ret = parse_stream_uri(&uri, &type, &ops); diff --git a/Pal/src/host/Linux-SGX/db_devices.c b/Pal/src/host/Linux-SGX/db_devices.c index d75fe795..5c9862ee 100644 --- a/Pal/src/host/Linux-SGX/db_devices.c +++ b/Pal/src/host/Linux-SGX/db_devices.c @@ -17,6 +17,7 @@ #include "pal_internal.h" #include "pal_linux.h" #include "pal_linux_error.h" +#include "toml.h" static int dev_open(PAL_HANDLE* handle, const char* type, const char* uri, int access, int share, int create, int options) { @@ -111,6 +112,19 @@ static int64_t dev_write(PAL_HANDLE handle, uint64_t offset, uint64_t size, cons return IS_ERR(bytes) ? unix_to_pal_error(ERRNO(bytes)) : bytes; } +static int dev_map(PAL_HANDLE handle, void** addr, int prot, uint64_t offset, uint64_t size) { + if (!IS_HANDLE_TYPE(handle, dev)) + return -PAL_ERROR_INVAL; + + if (handle->dev.fd == PAL_IDX_POISON) + return -PAL_ERROR_DENIED; + + assert(WITHIN_MASK(prot, PAL_PROT_MASK)); + + int ret = ocall_mmap_untrusted(handle->dev.fd, offset, size, PAL_PROT_TO_LINUX(prot), addr); + return IS_ERR(ret) ? unix_to_pal_error(ERRNO(ret)) : ret; +} + static int dev_close(PAL_HANDLE handle) { if (!IS_HANDLE_TYPE(handle, dev)) return -PAL_ERROR_INVAL; @@ -210,8 +224,696 @@ struct handle_ops g_dev_ops = { .open = &dev_open, .read = &dev_read, .write = &dev_write, + .map = &dev_map, .close = &dev_close, .flush = &dev_flush, .attrquery = &dev_attrquery, .attrquerybyhdl = &dev_attrquerybyhdl, }; + +/* + * Code below describes the deep-copy syntax in the TOML manifest used for copying complex nested + * objects out and in the SGX enclave. This syntax is currently used for IOCTL emulation. This + * syntax is generic enough to describe any memory layout for deep copy. + * + * The following example describes the main implementation details: + * + * struct pascal_str { uint8_t len; char str[]; }; + * struct c_str { char str[]; }; + * struct root { struct pascal_str* s1; struct c_str* s2; uint64_t s2_len; int8_t x; int8_t y; }; + * + * alignas(128) struct root obj; + * ioctl(devfd, _IOWR(DEVICE_MAGIC, DEVICE_FUNC, struct root), &obj); + * + * The example IOCTL takes as a third argument a pointer to an object of type `struct root` that + * contains two pointers to other objects (pascal-style string and a C-style string) and embeds two + * integers `x` and `y`. The two strings reside in separate memory regions in enclave memory. Note + * that the length of the C-style string is stored in the `s2_len` field of the root object. The + * `pascal_str` string is an input to the IOCTL, the `c_str` string is both input and output of the + * IOCTL, and the integers `x` and `y` are both outputs of the IOCTL. Also note that the root + * object is 128B-aligned (for illustration purposes). + * + * The corresponding deep-copy syntax in TOML looks like this: + * + * sgx.ioctl_structs.ROOT_FOR_DEVICE_FUNC = [ + * { align = 128, ptr = [ {name="pascal-str-len", size=1, type="out"}, + * {name="pascal-str", size="pascal-str-len", type="out"} ] }, + * { ptr = [ {name="c-str", size="c-str-len", type="inout"} ], size = 1 }, + * { name = "c-str-len", size = 8, unit = 1, type = "in" }, + * { onlyif = "c-str-len == pascal-str-len", size = 2, type = "in" } + * { onlyif = "c-str-len != pascal-str-len", size = 2, type = "none" } + * ] + * + * One can observe the following rules in this TOML syntax: + * 1. Each separate memory region is represented as a TOML array (`[]`). + * 2. Each sub-region of one memory region is represented as a TOML table (`{}`). + * 3. Each sub-region may be a pointer (`ptr`) to another memory region. In this case, the value of + * `ptr` is a TOML-array representation of that other memory region. The `ptr` sub-region always + * has size of 8B (assuming x86-64) and doesn't have an in/out type. The `size` field of the + * `ptr` sub-region has a different meaning than for non-pointer sub-regions: it is the number + * of adjacent memory regions that this pointer points to (i.e. it describes an array). + * 4. Sub-regions can be fixed-size (like the last sub-region containing two bytes `x` and `y`) or + * can be flexible-size (like the two strings). In the latter case, the `size` field contains a + * name of a sub-region where the actual size is stored. + * 5. Sub-regions that store the size of another sub-region must be 1, 2, 4, or 8 bytes in size. + * 6. Sub-regions may have a name for ease of identification; this is required for "size" + * sub-regions but may be omitted for all other kinds of sub-regions. + * 7. Sub-regions may have one of the four types: "out" to copy contents of the sub-region outside + * the enclave to untrusted memory, "in" to copy from untrusted memory to inside the enclave, + * "inout" to copy in both directions, "none" to not copy at all (useful for e.g. padding). + * Note that pointer sub-regions do not have a type. + * 8. The first sub-region (and only the first!) may specify the alignment of the memory region. + * 9. The total size of a sub-region is calculated as `size * unit`. By default `unit` is 1 byte. + * 10. Sub-regions may be conditioned using `onlyif = "simple boolean expression"`. In the example + * above, `x` and `y` will be copied back into the enclave only if the two strings have the + * same length; otherwise `x` and `y` will not be modified. The only currently supported format + * of expressions is "token1 == token2" or "token1 != token2", where `token1` and `token2` may + * be constant integers or sub region names. + * + * The diagram below shows how this complex object is copied from enclave memory (left side) to + * untrusted memory (right side). MR stands for "memory region", SR stands for "sub-region". Note + * how enclave pointers are copied and rewired to point to untrusted memory regions. + * + * struct root (MR1) | deep-copied struct (aligned at 128B) + * +------------------+ | +------------------------+ + * +----+ pascal_str* s1 | SR1 | +----+ pascal_str* s1 (MR1)| + * | | | | | | | + * | | c_str* s2 +-------+ SR2 | | | c_str* s2 +-------------+ + * | | | | | | | | | + * | | uint64_t s2_len | | SR3 | | | uint64_t s2_len | | + * | | | | | | | | | + * | | int8_t x, y | | SR4 | | | int8_t x=0, y=0 | | + * | +------------------+ | | | +------------------------+ | + * | | | +->| uint8_t len (MR2)| | + * v (MR2) | | | | | + * +-------------+ | | | char str[] | | + * | uint8_t len | | SR5 | +------------------------+ | + * | | | | | char str[] (MR3)|<-+ + * | char str[] | | SR6 | +------------------------+ + * +-------------+ | | + * (MR3) v | + * +----------+-+ | + * | char str[] | SR7 | + * +------------+ | + * + */ + +/* for simplicity and thread-safety, we allocate mem_regions and sub_regions on stack; we assume + * that deep copy of objects doesn't exceed the specified limits of memory and sub regions */ +#define MAX_MEM_REGIONS 32 +#define MAX_SUB_REGIONS 64 + +/* direction of copy: none (used for padding), out of enclave, inside enclave, both or a special + * "pointer" sub-region; default is COPY_NONE_ENCLAVE */ +enum mem_copy_type {COPY_NONE_ENCLAVE = 0, COPY_OUT_ENCLAVE, COPY_IN_ENCLAVE, COPY_INOUT_ENCLAVE, + COPY_PTR_ENCLAVE}; + +struct mem_region { + toml_array_t* toml_array; /* describes contigious sub_regions in this mem_region */ + void* encl_addr; /* base address of this memory region in enclave memory */ + bool adjacent; /* memory region adjacent to previous one? (used for arrays) */ +}; + +struct sub_region { + enum mem_copy_type type; /* direction of copy during OCALL (or pointer to another region) */ + char* name; /* may be NULL for unnamed regions */ + ssize_t align; /* alignment of this sub-region */ + ssize_t size; /* may be dynamically determined from another sub-region */ + char* size_name; /* needed if "size" sub region is defined after this sub region */ + ssize_t unit; /* total size in bytes is calculated as `size * unit` */ + void* encl_addr; /* base address of this sub region in enclave memory */ + void* untrusted_addr; /* base address of the corresponding sub region in untrusted memory */ + toml_array_t* mem_ptr; /* for pointers/arrays, specifies pointed-to mem region */ +}; + +/* finds a sub region with name `sub_region_name` among `sub_regions` and reads the value in it */ +static int get_sub_region_value(struct sub_region* sub_regions, int sub_regions_cnt, + const char* sub_region_name, ssize_t* value) { + /* it is important to iterate in reverse order because there may be an array of mem regions + * with same-named sub regions, and we want to find the "latest size" sub region, i.e. the one + * belonging to the same mem region */ + for (int i = sub_regions_cnt - 1; i >= 0; i--) { + if (sub_regions[i].name && !strcmp(sub_regions[i].name, sub_region_name)) { + /* found corresponding sub region, read its value */ + if (!sub_regions[i].encl_addr || sub_regions[i].encl_addr == (void*)-1) { + /* enclave address is invalid, user provided bad struct */ + return -EFAULT; + } + + if (sub_regions[i].size == sizeof(uint8_t)) { + *value = (ssize_t)(*((uint8_t*)sub_regions[i].encl_addr)); + } else if (sub_regions[i].size == sizeof(uint16_t)) { + *value = (ssize_t)(*((uint16_t*)sub_regions[i].encl_addr)); + } else if (sub_regions[i].size == sizeof(uint32_t)) { + *value = (ssize_t)(*((uint32_t*)sub_regions[i].encl_addr)); + } else if (sub_regions[i].size == sizeof(uint64_t)) { + *value = (ssize_t)(*((uint64_t*)sub_regions[i].encl_addr)); + } else { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (deep-copy sub-entry \'%s\' must be of " + "legitimate size: 1, 2, 4 or 8 bytes)\n", + sub_regions[i].name); + return -EINVAL; + } + + return 0; + } + } + + return -ENOENT; +} + +/* Parses a simple boolean expression in `expr`, finds corresponding values from sub regions and + * calculates the resulting boolean value. NOTE: currently supports only "x == y" and "x != y" + * where both "x" and "y" can be sub region names or constant integers. */ +static int calculate_boolean_expr(struct sub_region* sub_regions, int sub_regions_cnt, char* expr, + bool* value) { + int ret; + char* cur = expr; + + while (*cur == ' ' || *cur == '\t') { cur++; } + + /* read first token */ + char* token1 = cur; + while (isalnum(*cur) || *cur == '_' || *cur == '-') { cur++; } + size_t token1_len = cur - token1; + + while (*cur == ' ' || *cur == '\t') { cur++; } + + /* read comparator */ + char* compar = cur; + while (*cur == '=' || *cur == '!') { cur++; } + size_t compar_len = cur - compar; + + while (*cur == ' ' || *cur == '\t') { cur++; } + + /* read second token */ + char* token2 = cur; + while (isalnum(*cur) || *cur == '_' || *cur == '-') { cur++; } + size_t token2_len = cur - token2; + + while (*cur == ' ' || *cur == '\t') { cur++; } + + /* make sure the whole string is in format "token1 == token2" or "token1 != token2" */ + if (compar_len != 2 || (memcmp(compar, "==", 2) && (memcmp(compar, "!=", 2)))) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (the only supported comparators are \'==\' and " + "\'!=\' but they are not found in expression \'%s\')\n", expr); + return -EINVAL; + } + + if (*cur != '\0' || !token1_len || !token2_len) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (cannot parse expression \'%s\')\n", expr); + return -EINVAL; + } + + /* get actual values for two tokens */ + char* endptr = NULL; + token1[token1_len] = '\0'; + ssize_t val1 = strtol(token1, &endptr, /*base=*/0); + if (endptr != token1 + token1_len) { + /* could not read the constant integer, the token must be a string-name of a sub region */ + ret = get_sub_region_value(sub_regions, sub_regions_cnt, token1, &val1); + if (ret < 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (cannot find first sub region in expression " + "\'%s\')\n", expr); + return ret; + } + } + + token2[token2_len] = '\0'; + ssize_t val2 = strtol(token2, &endptr, /*base=*/0); + if (endptr != token2 + token2_len) { + /* could not read the constant integer, the token must be a string-name of a sub region */ + ret = get_sub_region_value(sub_regions, sub_regions_cnt, token2, &val2); + if (ret < 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (cannot find second sub region in expression " + "\'%s\')\n", expr); + return ret; + } + } + + if (!(memcmp(compar, "==", 2))) { + *value = val1 == val2; + } else { + *value = val1 != val2; + } + return 0; +} + +/* caller sets `_sub_regions_cnt` to maximum number of sub_regions; this variable is updated to + * return the number of actually used sub_regions */ +static int collect_sub_regions(toml_array_t* root_toml_array, void* root_encl_addr, + struct sub_region* sub_regions, int* _sub_regions_cnt) { + int ret; + + assert(root_toml_array && toml_array_nelem(root_toml_array) > 0); + assert(sub_regions && _sub_regions_cnt); + + int max_sub_regions = *_sub_regions_cnt; + int sub_regions_cnt = 0; + + for (int i = 0; i < max_sub_regions; i++) { + sub_regions[i].align = 0; + sub_regions[i].unit = 0; + sub_regions[i].size = -1; + sub_regions[i].name = NULL; + sub_regions[i].size_name = NULL; + sub_regions[i].encl_addr = NULL; + sub_regions[i].untrusted_addr = NULL; + sub_regions[i].mem_ptr = NULL; + } + + struct mem_region mem_regions[MAX_MEM_REGIONS] = {0}; + mem_regions[0].toml_array = root_toml_array; + mem_regions[0].encl_addr = root_encl_addr; + mem_regions[0].adjacent = false; + int mem_regions_cnt = 1; + + /* collecting memory regions and their sub-regions must use breadth-first search to dynamically + * calculate sizes of sub-regions even if they are specified via another sub-region's "name" */ + char* cur_encl_addr = NULL; /* char* type for pointer arithmetic */ + int mem_region_idx = 0; + while (mem_region_idx < mem_regions_cnt) { + struct mem_region* cur_mem_region = &mem_regions[mem_region_idx]; + mem_region_idx++; + + if (!cur_mem_region->adjacent) + cur_encl_addr = cur_mem_region->encl_addr; + + int cur_mem_region_first_sub_region = sub_regions_cnt; + + for (int i = 0; i < toml_array_nelem(cur_mem_region->toml_array); i++) { + toml_table_t* sub_region_info = toml_table_at(cur_mem_region->toml_array, i); + if (!sub_region_info) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (each memory subregion must be a TOML " + "table)\n"); + ret = -EINVAL; + goto out; + } + + if (sub_regions_cnt == max_sub_regions) { + SGX_DBG(DBG_E, "Too many memory sub-regions in a deep-copy syntax (maximum " + "possible is %d)\n", max_sub_regions); + ret = -ENOMEM; + goto out; + } + + struct sub_region* cur_sub_region = &sub_regions[sub_regions_cnt]; + sub_regions_cnt++; + + cur_sub_region->encl_addr = cur_encl_addr; + if (!cur_encl_addr || cur_encl_addr == (void*)-1) { + /* enclave address is invalid, user provided bad struct */ + ret = -EFAULT; + goto out; + } + + toml_raw_t sub_region_onlyif_raw = toml_raw_in(sub_region_info, "onlyif"); + toml_raw_t sub_region_name_raw = toml_raw_in(sub_region_info, "name"); + toml_raw_t sub_region_type_raw = toml_raw_in(sub_region_info, "type"); + toml_raw_t sub_region_align_raw = toml_raw_in(sub_region_info, "align"); + toml_raw_t sub_region_size_raw = toml_raw_in(sub_region_info, "size"); + toml_raw_t sub_region_unit_raw = toml_raw_in(sub_region_info, "unit"); + toml_array_t* sub_region_ptr_arr = toml_array_in(sub_region_info, "ptr"); + + if (sub_region_align_raw && i != 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'align\' may be specified only for the " + "first sub-region of the memory region)\n"); + ret = -EINVAL; + goto out; + } + + if (sub_region_type_raw && sub_region_ptr_arr) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'ptr\' sub-entries cannot specify " + "a \'type\'; pointers are never copied directly but rewired)\n"); + ret = -EINVAL; + goto out; + } + + if (sub_region_onlyif_raw) { + char* onlyif_expr = NULL; + ret = toml_rtos(sub_region_onlyif_raw, &onlyif_expr); + if (ret < 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'onlyif\' of a deep-copy sub-entry " + "must be a TOML string surrounded by double quotes)\n"); + ret = -EINVAL; + goto out; + } + + bool val = false; + /* "sub_regions_cnt - 1" is to exclude myself */ + ret = calculate_boolean_expr(sub_regions, sub_regions_cnt - 1, onlyif_expr, &val); + if (ret < 0) { + free(onlyif_expr); + goto out; + } + free(onlyif_expr); + + if (!val) { + /* onlyif expression is false, we must skip this sub region completely */ + sub_regions_cnt--; + continue; + } + } + + cur_sub_region->name = NULL; + if (sub_region_name_raw) { + ret = toml_rtos(sub_region_name_raw, &cur_sub_region->name); + if (ret < 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'name\' of a deep-copy sub-entry " + "must be a TOML string surrounded by double quotes)\n"); + ret = -EINVAL; + goto out; + } + } + + cur_sub_region->type = COPY_NONE_ENCLAVE; + if (sub_region_type_raw) { + char* type_str = NULL; + ret = toml_rtos(sub_region_type_raw, &type_str); + if (ret < 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'type\' of a deep-copy sub-entry " + "must be a TOML string surrounded by double quotes)\n"); + ret = -EINVAL; + goto out; + } + + if (!strcmp(type_str, "out")) { + cur_sub_region->type = COPY_OUT_ENCLAVE; + } + else if (!strcmp(type_str, "in")) { + cur_sub_region->type = COPY_IN_ENCLAVE; + } + else if (!strcmp(type_str, "inout")) { + cur_sub_region->type = COPY_INOUT_ENCLAVE; + } + else if (!strcmp(type_str, "none")) { + cur_sub_region->type = COPY_NONE_ENCLAVE; + } + else { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'type\' of a deep-copy sub-entry " + "must be one of \"out\", \"in\", \"inout\" or \"none\")\n"); + free(type_str); + ret = -EINVAL; + goto out; + } + + free(type_str); + } + + cur_sub_region->align = 0; + if (sub_region_align_raw) { + ret = toml_rtoi(sub_region_align_raw, &cur_sub_region->align); + if (ret < 0 || cur_sub_region->align <= 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'align\' of a deep-copy sub-entry " + "must be a positive number)\n"); + ret = -EINVAL; + goto out; + } + } + + if (sub_region_ptr_arr) { + /* only set type for now, we postpone pointer/array handling for later */ + cur_sub_region->type = COPY_PTR_ENCLAVE; + cur_sub_region->mem_ptr = sub_region_ptr_arr; + } + + cur_sub_region->size = -1; + if (sub_region_size_raw) { + ret = toml_rtos(sub_region_size_raw, &cur_sub_region->size_name); + if (ret == 0) { + ssize_t val = -1; + /* "sub_regions_cnt - 1" is to exclude myself; do not fail if couldn't find + * (we will try later one more time) */ + ret = get_sub_region_value(sub_regions, sub_regions_cnt - 1, + cur_sub_region->size_name, &val); + if (ret < 0 && ret != -ENOENT) { + goto out; + } + cur_sub_region->size = val; + } else { + /* size is specified not as string (another sub-region's name), then must be + * specified explicitly as number of bytes */ + ret = toml_rtoi(sub_region_size_raw, &cur_sub_region->size); + if (ret < 0 || cur_sub_region->size <= 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'size\' of a deep-copy " + "sub-entry must be a TOML string or a positive number)\n"); + ret = -EINVAL; + goto out; + } + } + } + + cur_sub_region->unit = 1; /* 1 byte by default */ + if (sub_region_unit_raw) { + ret = toml_rtoi(sub_region_unit_raw, &cur_sub_region->unit); + if (ret < 0 || cur_sub_region->unit <= 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'unit\' of a deep-copy sub-entry " + "must be a positive number)\n"); + ret = -EINVAL; + goto out; + } + } + cur_sub_region->size *= cur_sub_region->unit; + cur_encl_addr += cur_sub_region->type == COPY_PTR_ENCLAVE ? 8 : cur_sub_region->size; + } + + /* iterate through collected pointer/array sub regions and add corresponding mem regions */ + for (int i = cur_mem_region_first_sub_region; i < sub_regions_cnt; i++) { + if (sub_regions[i].type != COPY_PTR_ENCLAVE) + continue; + + if (sub_regions[i].size >= 0) { + /* sizes was found in the first swoop, nothing to do here */ + } else if (sub_regions[i].size < 0 && sub_regions[i].size_name) { + /* pointer/array size was not found in the first swoop, try again */ + ssize_t val = -1; + ret = get_sub_region_value(sub_regions, sub_regions_cnt, sub_regions[i].size_name, + &val); + if (ret < 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (cannot find sub region \'%s\')\n", + sub_regions[i].size_name); + goto out; + } + sub_regions[i].size = val; + } else { + /* size is not specified at all for this sub region, assume it is 1 */ + sub_regions[i].size = 1; + } + + for (ssize_t k = 0; k < sub_regions[i].size; k++) { + if (mem_regions_cnt == MAX_MEM_REGIONS) { + SGX_DBG(DBG_E, "Too many memory regions in a deep-copy syntax (maximum " + "possible is %d)\n", MAX_MEM_REGIONS); + ret = -ENOMEM; + goto out; + } + + void* mem_region_addr = *((void**)sub_regions[i].encl_addr); + if (!mem_region_addr) + continue; + + mem_regions[mem_regions_cnt].toml_array = sub_regions[i].mem_ptr; + mem_regions[mem_regions_cnt].encl_addr = mem_region_addr; + mem_regions[mem_regions_cnt].adjacent = k > 0; + mem_regions_cnt++; + } + + sub_regions[i].size = sizeof(void*); /* rewire to actual size of sub-region */ + } + } + + *_sub_regions_cnt = sub_regions_cnt; + ret = 0; +out: +#if 0 + for (int i = 0; i < sub_regions_cnt; i++) { + /* we print 4B memory cells in hope that this memory cell is accessible (only for debug) */ + uint32_t value = sub_regions[i].encl_addr ? *((uint32_t*)sub_regions[i].encl_addr) : 0; + SGX_DBG(DBG_E, "===== sub_region[%d]: \'%s\', size=%ld, addr=%p, value=%u\n", + i, sub_regions[i].name, sub_regions[i].size, sub_regions[i].encl_addr, value); + } +#endif + for (int i = 0; i < sub_regions_cnt; i++) { + if (ret == 0) { + /* misc sanity checks on all collected sub regions */ + if (sub_regions[i].size < 0) { + SGX_DBG(DBG_E, "Invalid deep-copy syntax (\'size\' is unspecified/negative in sub " + "region \'%s\')\n", sub_regions[i].name); + ret = -EINVAL; + } + + if (sub_regions[i].size > 0 && (!sub_regions[i].encl_addr || + sub_regions[i].encl_addr == (void*)-1)) { + /* enclave address is invalid, user provided bad struct */ + ret = -EFAULT; + } + } + + /* "name" fields are not needed after we collected all sub_regions */ + free(sub_regions[i].name); + free(sub_regions[i].size_name); + sub_regions[i].name = NULL; + sub_regions[i].size_name = NULL; + } + return ret; +} + +static void copy_sub_regions_to_untrusted(struct sub_region* sub_regions, int sub_regions_cnt, + void* untrusted_addr) { + char* cur_untrusted_addr = untrusted_addr; /* char* type for pointer arithmetic */ + for (int i = 0; i < sub_regions_cnt; i++) { + if (sub_regions[i].align > 0) + cur_untrusted_addr = ALIGN_UP_PTR(cur_untrusted_addr, sub_regions[i].align); + + sub_regions[i].untrusted_addr = cur_untrusted_addr; + if (sub_regions[i].type == COPY_OUT_ENCLAVE || sub_regions[i].type == COPY_INOUT_ENCLAVE) + memcpy(cur_untrusted_addr, sub_regions[i].encl_addr, sub_regions[i].size); + cur_untrusted_addr += sub_regions[i].size; + } + + for (int i = 0; i < sub_regions_cnt; i++) { + if (sub_regions[i].type == COPY_PTR_ENCLAVE) { + void* encl_ptr_value = *((void**)sub_regions[i].encl_addr); + /* rewire pointer value in untrusted memory to a corresponding untrusted sub-region */ + for (int j = 0; j < sub_regions_cnt; j++) { + if (sub_regions[j].encl_addr == encl_ptr_value) { + *((void**)sub_regions[i].untrusted_addr) = sub_regions[j].untrusted_addr; + break; + } + } + } + } +} + +static void copy_sub_regions_to_enclave(struct sub_region* sub_regions, int sub_regions_cnt) { + for (int i = 0; i < sub_regions_cnt; i++) { + if (sub_regions[i].type == COPY_IN_ENCLAVE || sub_regions[i].type == COPY_INOUT_ENCLAVE) + memcpy(sub_regions[i].encl_addr, sub_regions[i].untrusted_addr, sub_regions[i].size); + } +} + +int _DkDeviceIoControl(PAL_HANDLE handle, unsigned int cmd, uint64_t arg) { + int ret; + + if (!IS_HANDLE_TYPE(handle, dev)) + return -PAL_ERROR_INVAL; + + if (handle->dev.fd == PAL_IDX_POISON) + return -PAL_ERROR_DENIED; + + toml_table_t* manifest_sgx = toml_table_in(g_pal_state.manifest_root, "sgx"); + if (!manifest_sgx) + return -PAL_ERROR_NOTIMPLEMENTED; + + toml_table_t* toml_allowed_ioctls = toml_table_in(manifest_sgx, "allowed_ioctls"); + if (!toml_allowed_ioctls) + return -PAL_ERROR_NOTIMPLEMENTED; + + ssize_t toml_allowed_ioctls_cnt = toml_table_ntab(toml_allowed_ioctls); + if (toml_allowed_ioctls_cnt <= 0) + return -PAL_ERROR_NOTIMPLEMENTED; + + for (ssize_t i = 0; i < toml_allowed_ioctls_cnt; i++) { + const char* toml_allowed_ioctl_key = toml_key_in(toml_allowed_ioctls, i); + assert(toml_allowed_ioctl_key); + + toml_table_t* toml_ioctl_table = toml_table_in(toml_allowed_ioctls, toml_allowed_ioctl_key); + if (!toml_ioctl_table) + continue; + + toml_raw_t toml_ioctl_request_raw = toml_raw_in(toml_ioctl_table, "request"); + if (!toml_ioctl_request_raw) + continue; + + int64_t ioctl_request = 0x0; + ret = toml_rtoi(toml_ioctl_request_raw, &ioctl_request); + if (ret < 0 || ioctl_request == 0x0) { + SGX_DBG(DBG_E, "Invalid request value of allowed ioctl \'%s\' in manifest\n", + toml_allowed_ioctl_key); + continue; + } + + if (ioctl_request == (int64_t)cmd) { + /* found this IOCTL request in the manifest, now must find the corresponding struct */ + toml_raw_t toml_ioctl_struct_raw = toml_raw_in(toml_ioctl_table, "struct"); + if (!toml_ioctl_struct_raw) { + SGX_DBG(DBG_E, "Cannot find struct value of allowed ioctl \'%s\' in manifest\n", + toml_allowed_ioctl_key); + return -PAL_ERROR_NOTIMPLEMENTED; + } + + char* ioctl_struct_str = NULL; + ret = toml_rtos(toml_ioctl_struct_raw, &ioctl_struct_str); + if (ret < 0) { + SGX_DBG(DBG_E, "Invalid struct value of allowed ioctl \'%s\' in manifest " + "(sgx.allowed_ioctls.[identifier].struct must be a TOML string)\n", + toml_allowed_ioctl_key); + return -PAL_ERROR_INVAL; + } + + toml_table_t* toml_ioctl_structs = toml_table_in(manifest_sgx, "ioctl_structs"); + if (!toml_ioctl_structs) { + SGX_DBG(DBG_E, "There are no ioctl structs found in manifest\n"); + free(ioctl_struct_str); + return -PAL_ERROR_INVAL; + } + + toml_array_t* toml_ioctl_struct = toml_array_in(toml_ioctl_structs, ioctl_struct_str); + if (!toml_ioctl_struct) { + SGX_DBG(DBG_E, "Cannot find struct value \'%s\' of allowed ioctl \'%s\' in " + "manifest (or it is not a correctly formatted TOML array)\n", + ioctl_struct_str, toml_allowed_ioctl_key); + free(ioctl_struct_str); + return -PAL_ERROR_INVAL; + } + free(ioctl_struct_str); + + if (toml_array_nelem(toml_ioctl_struct) == 0) { + /* special case of an empty TOML array == base-type or ignored IOCTL argument */ + ret = ocall_ioctl(handle->dev.fd, cmd, arg); + return ret < 0 ? unix_to_pal_error(ERRNO(ret)) : 0; + } + + /* typical case of used IOCTL argument: deep-copy the IOCTL argument's input data + * outside of enclave, execute the IOCTL OCALL, and deep-copy the IOCTL argument's + * output data back into enclave */ + struct sub_region sub_regions[MAX_SUB_REGIONS]; + int sub_regions_cnt = MAX_SUB_REGIONS; + ret = collect_sub_regions(toml_ioctl_struct, (void*)arg, sub_regions, &sub_regions_cnt); + if (ret < 0) { + if (ret != -EFAULT) { + SGX_DBG(DBG_E, "Invalid struct format of allowed ioctl \'%s\' in manifest\n", + toml_allowed_ioctl_key); + } + return unix_to_pal_error(ERRNO(ret)); + } + + void* untrusted_addr = NULL; + size_t untrusted_size = 0; + for (int i = 0; i < sub_regions_cnt; i++) { + assert(sub_regions[i].size > 0); + untrusted_size += sub_regions[i].size + sub_regions[i].align; + } + + ret = ocall_mmap_untrusted(/*fd=*/-1, /*offset=*/0, untrusted_size, + PROT_READ | PROT_WRITE, &untrusted_addr); + if (ret < 0) { + return -PAL_ERROR_NOMEM; + } + + copy_sub_regions_to_untrusted(sub_regions, sub_regions_cnt, untrusted_addr); + + ret = ocall_ioctl(handle->dev.fd, cmd, (uint64_t)untrusted_addr); + if (ret < 0) { + ocall_munmap_untrusted(untrusted_addr, untrusted_size); + return unix_to_pal_error(ERRNO(ret)); + } + + copy_sub_regions_to_enclave(sub_regions, sub_regions_cnt); + ocall_munmap_untrusted(untrusted_addr, untrusted_size); + return 0; + } + } + + return -PAL_ERROR_NOTIMPLEMENTED; +} diff --git a/Pal/src/host/Linux-SGX/db_eventfd.c b/Pal/src/host/Linux-SGX/db_eventfd.c index 881de862..9885be01 100644 --- a/Pal/src/host/Linux-SGX/db_eventfd.c +++ b/Pal/src/host/Linux-SGX/db_eventfd.c @@ -48,8 +48,14 @@ static int eventfd_pal_open(PAL_HANDLE* handle, const char* type, const char* ur return -PAL_ERROR_INVAL; } - /* Using create arg as a work-around (note: initval is uint32 but create is int32).*/ - ret = ocall_eventfd(create, eventfd_type(options)); + if (options & PAL_OPTION_EFD_SEMAPHORE) { + /* FIXME: semaphore option is abused to hint that `count` already contains host FD; + * currently used for LibOS emulation of DRM_IOCTL_I915_GEM_EXECBUFFER2_WR ioctl */ + ret = create; + } else { + /* Using create arg as a work-around (note: initval is uint32 but create is int32).*/ + ret = ocall_eventfd(create, eventfd_type(options)); + } if (IS_ERR(ret)) return unix_to_pal_error(ERRNO(ret)); diff --git a/Pal/src/host/Linux-SGX/enclave_ocalls.c b/Pal/src/host/Linux-SGX/enclave_ocalls.c index c74a429b..0774cc7f 100644 --- a/Pal/src/host/Linux-SGX/enclave_ocalls.c +++ b/Pal/src/host/Linux-SGX/enclave_ocalls.c @@ -1580,3 +1580,24 @@ int ocall_sched_getaffinity(void* tcs, size_t cpumask_size, void* cpu_mask) { sgx_reset_ustack(old_ustack); return retval; } + +int ocall_ioctl(int fd, unsigned int cmd, unsigned long arg) { + int retval = 0; + ms_ocall_ioctl_t* ms; + + void* old_ustack = sgx_prepare_ustack(); + ms = sgx_alloc_on_ustack_aligned(sizeof(*ms), alignof(*ms)); + if (!ms) { + sgx_reset_ustack(old_ustack); + return -EPERM; + } + + WRITE_ONCE(ms->ms_fd, fd); + WRITE_ONCE(ms->ms_cmd, cmd); + WRITE_ONCE(ms->ms_arg, arg); + + retval = sgx_exitless_ocall(OCALL_IOCTL, ms); + + sgx_reset_ustack(old_ustack); + return retval; +} diff --git a/Pal/src/host/Linux-SGX/enclave_ocalls.h b/Pal/src/host/Linux-SGX/enclave_ocalls.h index 6e37e1a5..532a07e0 100644 --- a/Pal/src/host/Linux-SGX/enclave_ocalls.h +++ b/Pal/src/host/Linux-SGX/enclave_ocalls.h @@ -96,6 +96,8 @@ int ocall_update_debugger(struct debug_map* _Atomic* debug_map); int ocall_eventfd(unsigned int initval, int flags); +int ocall_ioctl(int fd, unsigned int cmd, unsigned long arg); + /*! * \brief Execute untrusted code in PAL to obtain a quote from the Quoting Enclave. * diff --git a/Pal/src/host/Linux-SGX/ocall_types.h b/Pal/src/host/Linux-SGX/ocall_types.h index dc23e38d..a7c781fd 100644 --- a/Pal/src/host/Linux-SGX/ocall_types.h +++ b/Pal/src/host/Linux-SGX/ocall_types.h @@ -62,6 +62,7 @@ enum { OCALL_DELETE, OCALL_UPDATE_DEBUGGER, OCALL_EVENTFD, + OCALL_IOCTL, OCALL_GET_QUOTE, OCALL_NR, }; @@ -290,6 +291,12 @@ typedef struct { int ms_flags; } ms_ocall_eventfd_t; +typedef struct { + int ms_fd; + unsigned int ms_cmd; + unsigned long ms_arg; +} ms_ocall_ioctl_t; + typedef struct { bool ms_is_epid; sgx_spid_t ms_spid; diff --git a/Pal/src/host/Linux-SGX/sgx_enclave.c b/Pal/src/host/Linux-SGX/sgx_enclave.c index 81dff515..45c254d0 100644 --- a/Pal/src/host/Linux-SGX/sgx_enclave.c +++ b/Pal/src/host/Linux-SGX/sgx_enclave.c @@ -682,6 +682,13 @@ static long sgx_ocall_eventfd(void* pms) { return ret; } +static long sgx_ocall_ioctl(void* pms) { + ms_ocall_ioctl_t* ms = (ms_ocall_ioctl_t*)pms; + ODEBUG(OCALL_IOCTL, ms); + long ret = INLINE_SYSCALL(ioctl, 3, ms->ms_fd, ms->ms_cmd, ms->ms_arg); + return ret; +} + static long sgx_ocall_update_debugger(void* pms) { ms_ocall_update_debugger_t* ms = (ms_ocall_update_debugger_t*)pms; ODEBUG(OCALL_UPDATE_DEBUGGER, ms); @@ -742,6 +749,7 @@ sgx_ocall_fn_t ocall_table[OCALL_NR] = { [OCALL_DELETE] = sgx_ocall_delete, [OCALL_UPDATE_DEBUGGER] = sgx_ocall_update_debugger, [OCALL_EVENTFD] = sgx_ocall_eventfd, + [OCALL_IOCTL] = sgx_ocall_ioctl, [OCALL_GET_QUOTE] = sgx_ocall_get_quote, }; diff --git a/Pal/src/host/Linux/db_devices.c b/Pal/src/host/Linux/db_devices.c index 97b9d34d..7468b5b3 100644 --- a/Pal/src/host/Linux/db_devices.c +++ b/Pal/src/host/Linux/db_devices.c @@ -110,6 +110,26 @@ static int64_t dev_write(PAL_HANDLE handle, uint64_t offset, uint64_t size, cons return IS_ERR(bytes) ? unix_to_pal_error(ERRNO(bytes)) : bytes; } +static int dev_map(PAL_HANDLE handle, void** addr, int prot, uint64_t offset, uint64_t size) { + if (!IS_HANDLE_TYPE(handle, dev)) + return -PAL_ERROR_INVAL; + + if (handle->dev.fd == PAL_IDX_POISON) + return -PAL_ERROR_DENIED; + + assert(WITHIN_MASK(prot, PAL_PROT_MASK)); + + void* mem = *addr; + int flags = MAP_FILE | PAL_MEM_FLAGS_TO_LINUX(0, prot) | (mem ? MAP_FIXED : 0); + + mem = (void*)ARCH_MMAP(mem, size, PAL_PROT_TO_LINUX(prot), flags, handle->dev.fd, offset); + if (IS_ERR_P(mem)) + return unix_to_pal_error(ERRNO_P(mem)); + + *addr = mem; + return 0; +} + static int dev_close(PAL_HANDLE handle) { if (!IS_HANDLE_TYPE(handle, dev)) return -PAL_ERROR_INVAL; @@ -201,8 +221,20 @@ struct handle_ops g_dev_ops = { .open = &dev_open, .read = &dev_read, .write = &dev_write, + .map = &dev_map, .close = &dev_close, .flush = &dev_flush, .attrquery = &dev_attrquery, .attrquerybyhdl = &dev_attrquerybyhdl, }; + +int _DkDeviceIoControl(PAL_HANDLE handle, unsigned int cmd, uint64_t arg) { + if (!IS_HANDLE_TYPE(handle, dev)) + return -PAL_ERROR_INVAL; + + if (handle->dev.fd == PAL_IDX_POISON) + return -PAL_ERROR_DENIED; + + int ret = INLINE_SYSCALL(ioctl, 3, handle->dev.fd, cmd, arg); + return IS_ERR(ret) ? unix_to_pal_error(ERRNO(ret)) : 0; +} diff --git a/Pal/src/host/Linux/db_eventfd.c b/Pal/src/host/Linux/db_eventfd.c index 06260460..5410fbde 100644 --- a/Pal/src/host/Linux/db_eventfd.c +++ b/Pal/src/host/Linux/db_eventfd.c @@ -50,8 +50,14 @@ static int eventfd_pal_open(PAL_HANDLE* handle, const char* type, const char* ur return -PAL_ERROR_INVAL; } - /* Using create arg as a work-around (note: initval is uint32 but create is int32).*/ - ret = INLINE_SYSCALL(eventfd2, 2, create, eventfd_type(options)); + if (options & PAL_OPTION_EFD_SEMAPHORE) { + /* FIXME: semaphore option is abused to hint that `count` already contains host FD; + * currently used for LibOS emulation of DRM_IOCTL_I915_GEM_EXECBUFFER2_WR ioctl */ + ret = create; + } else { + /* Using create arg as a work-around (note: initval is uint32 but create is int32).*/ + ret = INLINE_SYSCALL(eventfd2, 2, create, eventfd_type(options)); + } if (IS_ERR(ret)) return unix_to_pal_error(ERRNO(ret)); diff --git a/Pal/src/host/Skeleton/db_devices.c b/Pal/src/host/Skeleton/db_devices.c index 15685acb..a7e96ff5 100644 --- a/Pal/src/host/Skeleton/db_devices.c +++ b/Pal/src/host/Skeleton/db_devices.c @@ -25,6 +25,10 @@ static int64_t dev_write(PAL_HANDLE handle, uint64_t offset, uint64_t size, cons return -PAL_ERROR_NOTIMPLEMENTED; } +static int dev_map(PAL_HANDLE handle, void** addr, int prot, uint64_t offset, uint64_t size) { + return -PAL_ERROR_NOTIMPLEMENTED; +} + static int dev_close(PAL_HANDLE handle) { return -PAL_ERROR_NOTIMPLEMENTED; } @@ -45,8 +49,16 @@ struct handle_ops g_dev_ops = { .open = &dev_open, .read = &dev_read, .write = &dev_write, + .map = &dev_map, .close = &dev_close, .flush = &dev_flush, .attrquery = &dev_attrquery, .attrquerybyhdl = &dev_attrquerybyhdl, }; + +int _DkDeviceIoControl(PAL_HANDLE handle, unsigned int cmd, uint64_t arg) { + __UNUSED(handle); + __UNUSED(cmd); + __UNUSED(arg); + return -PAL_ERROR_NOTIMPLEMENTED; +} diff --git a/Pal/src/pal-symbols b/Pal/src/pal-symbols index be549ca7..01a43eeb 100644 --- a/Pal/src/pal-symbols +++ b/Pal/src/pal-symbols @@ -45,6 +45,7 @@ DkStreamAttributesSetByHandle DkMemoryAvailableQuota DkDebugAttachBinary DkDebugDetachBinary +DkDeviceIoControl DkAttestationReport DkAttestationQuote DkSetProtectedFilesKey diff --git a/Pal/src/pal_internal.h b/Pal/src/pal_internal.h index 4830d8dd..f613b35e 100644 --- a/Pal/src/pal_internal.h +++ b/Pal/src/pal_internal.h @@ -282,6 +282,7 @@ int _DkSegmentRegisterSet(int reg, const void* addr); int _DkSegmentRegisterGet(int reg, void** addr); int _DkInstructionCacheFlush(const void* addr, int size); int _DkCpuIdRetrieve(unsigned int leaf, unsigned int subleaf, unsigned int values[4]); +int _DkDeviceIoControl(PAL_HANDLE handle, unsigned int cmd, uint64_t arg); int _DkAttestationReport(PAL_PTR user_report_data, PAL_NUM* user_report_data_size, PAL_PTR target_info, PAL_NUM* target_info_size, PAL_PTR report, PAL_NUM* report_size); diff --git a/python/graphenelibos/sgx_sign.py b/python/graphenelibos/sgx_sign.py index 6ca8886c..52670536 100644 --- a/python/graphenelibos/sgx_sign.py +++ b/python/graphenelibos/sgx_sign.py @@ -71,6 +71,12 @@ def read_manifest(filename): manifest_layout.append((None, None)) break + if line.strip().startswith("{") or line.strip().startswith("}") or \ + line.strip().startswith("[") or line.strip().startswith("]"): + # TOML multi-line syntax, add it as-is + manifest_layout.append((None, line)) + continue + pound = line.find("#") if pound != -1: comment = line[pound:].strip()