Sunday, 21 August 2016

[Gsoc] [ lowRISC] A Trusted Execution Environment on seL4 - Full Report

In this blog post, I will be talking about my Google Summer of Code 2016 project with lowRISC and will include a walkthrough on installation and usage as well. My project was to build a standalone open source project, A trusted execution environment called "tee-os" which models the functionality of OPTEE but can be portable across architectures.


lowRISC is an open source organization who is aiming to create a fully open source System on chip computer based on the RISC-V architecture.

My project for Google summer of code 2016 was to build a Trusted Execution Environment which models the features and architecture of OPTEE by Linaro and also is compliant to the Global Platform standards.

RISC-V platform saw a port of seL4 operating system in the previous Google Summer of Code, So I built the Trusted Execution Environment on top of the seL4 kernel so that I could use the microkernel as a base and also encourage developers as an Architecture independent solution for a TEE would be mutual beneficial for all.

I was mentored by


seL4 is a formally verified kernel with complete end to end proofs, therefore it is suitable for building a system which can be trusted.

The seL4 kernel provides very limited functionality like IPC's and memory management  in accordance to microkernel design, which is the reason for its secure nature.

seL4 kernel:
The sel4 kernel provides basic functionality like IPC and memory management. It also allows two threads/processes to be completely isolated with the concept of capability spaces.

tee-os container: This is the root task of the system, this is loaded first and basically spawns the rich-os and Trusted applications in separate isolated threads. It allows manages communicated between the client and the trusted app.

Rich-OS: rich os is an operating system with a wealth features eg. Linux, Android. For the sake if this project the rich-os is just a simple stub os. This is preferably virtualized in the real world. A hypervisor would sit above this level in that case.

client app: The client app is the client facing part of the application It is started in the rich-os , it makes use of tee_client_api to communicate with the Trusted application/tee-container.

trusted application: The trusted application sits in an isolated container and has access to the tee_internal_api to communicate with the tee-container/client app and the tee_crpyto_api which provides cryptographic operations.


This section will explain the features of TEE-OS. The design principles and features have been heavily inspired by OPTEE-OS and are in compliance with the GlobalPlatform standard for TEE's. The process of writing client and secure apps for seL4 TEE-OS is very similar to OPTEE-OS. A number of functions need to be implemented on both the components and function names and TA names are provided in config files.

void encrypt(char *arr){
 uint8_t key[16];
 uint8_t plaintext[16];
 uint8_t ciphertext[16];
 int param_len = strlen(arr);
 char plaintext_string[16 * ceil_num(param_len/16.0)]; 
 for(int i= strlen(arr); i < strlen(plaintext_string);++i){
  plaintext_string[i] = 0;
 AES128_STRING_parse(key_string, key);
 AES128_STRING_parse(plaintext_string, plaintext);
 AES128_ECB_encrypt(plaintext, key, ciphertext);
 int i,j=0;
 for (i = 0; i < 16; i++){
  sprintf(arr+j,"%02x", ciphertext[i]);

void decrypt(char *arr){
 uint8_t key[16];
 uint8_t plaintext[16];
 uint8_t ciphertext[16];
 int param_len = strlen(arr);
 char plaintext_string[16 * ceil_num(param_len/16.0)];
 for(int i= strlen(arr); i < strlen(plaintext_string);++i){
  plaintext_string[i] = 0;
 AES128_STRING_parse(key_string, key);  
 AES128_STRING_parse(plaintext_string, plaintext);
 AES128_ECB_decrypt(plaintext, key, ciphertext);
 int i,j=0;
 for (i = 0; i < 16; i++){
  sprintf(arr+j,"%02x", ciphertext[i]);

//Every ta must have a function handler.
tee_result function_handler(int msg, int func_id ,seL4_Word* param_arr,int length){
 if(func_id == HELLO_TA_INCREMENT ){
  return tee_make_result(increment(msg),NULL,0);
 }else if(func_id == HELLO_TA_DECRYPT){
  char *temp = (char *)param_arr;
  return tee_make_result(0,temp,length * sizeof(seL4_Word));  
 else if(func_id == HELLO_TA_ENCRYPT){
  char *temp = (char *)param_arr;
  return tee_make_result(0,temp,length * sizeof(seL4_Word));  
 }else if(func_id == HELLO_TA_DECRYPT){
  char *temp = (char *)param_arr;
  return tee_make_result(0,temp,length * sizeof(seL4_Word));
  return tee_ta_failure();

The function handler is similar to the OPTEE entry point function, It calls respective functions based on ID.

Some features of TEE-OS are:

  1. Multiple TA support.
  2. Cryptographic library with AES and hash algorithms
  3. Multiple function call interface
  4. Portable to other architectures
  5. Runs on RISC-V (experimental) : not everything works yet

Github Repos:


For building sel4 based systems the following packages are necessary
sudo add-apt-repository universe
sudo apt-get install -y python-software-properties
sudo apt-get update

sudo apt-get install -y build-essential realpath
sudo apt-get install -y gcc-multilib ccache ncurses-dev
sudo apt-get install -y gcc-arm-linux-gnueabi
sudo apt-get install -y gcc-arm-none-eabi

sudo apt-get install -y python-pip python-jinja2 python-ply  python-tempita
sudo pip install --upgrade pip
sudo pip install pyelftools

Toolchains and cross-compilers:

sudo apt-get update
sudo apt-get install build-essential realpath
sudo apt-get install gcc-multilib ccache ncurses-dev
sudo add-apt-repository universe
sudo apt-get update
sudo apt-get install gcc-arm-linux-gnueabi
sudo apt-get install gcc-arm-none-eabi

Miscellaneous tools:

sudo apt-get install qemu-system-arm qemu-system-x86
sudo apt-get install git phablet-tools

If you want to build it for RISC-V you will require risc-v-gnu-toolchain and spike emulator . sel4 branch for RISC-V is available at link.

Installing tee-os:
The following steps download and install tee-os along with a sample client and trusted app. The manifest fest contains links to all other sub modules.

repo init -u -m tee-os.xml
repo sync
make kzm_debug_xml_defconfig
make silentoldconfig
make simulate-kzm

The sample app demonstrates a client calling the TA(trusted app) for three functions, firstly it sends a string to be encrypted, then it passes the encrypted string to be decrypted. The third function just increments a number in the secure world.. The result is shown below.

Some notable achievements are,

  • Built proof of concept system which shows the working of a sample trusted app which runs isolated from the operating system.
  • Implemented client and internal libraries which have the same interface as OPTEE-OS libraries
  • Implemented some cryptographic libraries like AES as a part of crypto api.
  • Built it for ARM as well as RISC-V and have successfully tested it out.
  • Good starting point for a future collaborated project.

Future work:

There is a lot of room for future work in this project, Firstly an important component of TEE's is secure storage, secure storage is used to store cryptographic keys and ciphers. Sadly seL4 did not have support for file systems at the moment, Secure storage should be implemented with the help of a client OS like linux.

The seL4 build for RISC-V is still experimental as some libraries like muslc for RISC-V have not been ported fully yet.

More people must be encouraged to contribute to this project as an architecture independent TEE is beneficial for everyone. Also since I built this project from scratch I haven't been able to perform detailed tests. Writing a test package is also necessary.

Saturday, 18 June 2016

Trusted Execution Environment on seL4 [UPDATE 2]

In this post I will explain my progress since the previous update and also provide some high level insights on how the architecture will look like.

On speaking with my mentors Stefan and Hesham we decided that it would be nice to have a fully working proof of concept of the TEE and then add depth in the form of libraries.

A high level flowchart of the execution would look something like this

Some properties would be:
  • The tee-container would be the main process which boots up first, it spawns the rich os and the ta's
  • All the components have different CSpace and Vspace hence they are isolated from each other.
  • For the sake of demonstration the Rich OS and client apps are very simple programs used just to test the communication.
I have built all three components and have built simple API's for the client and the TA similar to what OPTEE provides.

The sample TA provides two functions , one for encrypting a string and the other for incrementing a number

The client will look like this :

The output is:

For the sake of demonstration the encrypt function on the TA is a simple caesar's cipher , Later on I will be porting the libtomcrypt library.

The features which work now are:
  • Communication from client-os and isolation of all components
  • passing and receiving any type of complex parameters
  • multiple-TA (not tested extensively)
  • multiple function calls to TA
  • Client and TA libraries
I can conclude that the skeleton and structure for the seL4 TEE-OS is ready.
The tasks for the future are:
  • Study GlobalPlatform specifications and make the sel4-tee-os in compliance to that.
  • provide api's for secure-storage and crypto
  • try sel4-tee-os on sel4 port of risc-v


Friday, 27 May 2016

Trusted Execution Environment on seL4 [Update 1]

The coding period for this year's Google Summer Of Code started earlier this week and I take this as an opportunity to write about my plan for the project and report on the progress so far. As I had written in my earlier blog post , the key to building a TEE for RISC-V lies in the seL4 port for it. So my approach will involve building a TEE on top of seL4. Instead of going for para-virtualized solutions of porting OPTEE-OS I have decided to implement the features of OPTEE as a library for seL4 and build a TEE which for the most parts can be architecture independent.

Architecture diagram:


seL4 layer: This is the standard implementation of seL4 ,for now I have been using ia32 implementation but I could switch to the RISC-V one once it is stable.

virtualized RICH-OS: A virtualized OS like Linux or Android on top of a VMM. It will have a client library and have drivers to communicate with the TEE-container.

TEE container: The TEE container is similar to OPTEE-OS it will house the Trusted applications and allow communication between the RICH-OS and the TA.

TA: Trusted applications will be spawned as processes from the TEE-container and the library will be similar to OPTEE's tee_internal_api . There will be a format which a TA has to follow quite similar to OPTEE's format.

To be certified by Global Platform a TEE should provide the following services.

  • Trusted Storage API for data and keys
  • Time API services
  • Cryptographic operations
  • Arithmetic API
Another feature which is essential in a TEE is secure boot. I need to implement a secure boot features which guarantees booting into the Trusted-OS by a chain of trust.

Progress so far:
The github repository for the project:
and for the sample app :

I have built a proof of concept TEE-container and a sample app quite similar to the LCU14 OPTEE Hello World demo

For now the TEE-container boots up and is able to start Trusted applications, it is also able to call functions on the TA.
The internal api for the TA allows for a function handler to be defined to handle function calls just like in OPTEE.
The sample TA has a function which increments the input parameter just like LCU14 OPTEE demo.

Writing the TA is quite similar to the process followed for OPTEE.

Next weeks plan:
  1. Support multiple TA's
  2. Each TA should have memory isolation.
  3. Look at implementing some services if time permits.

I will most likely make this blog weekly, please feel free to share your opinions and questions on this project.

Sunday, 8 May 2016

[GSOC 2016 ] Implementing a Trusted Execution Environment for RISC-V architecture

I am happy to announce that I will be participating in the 2016 edition of the Google Summer Of Code with lowRISC[1] ,an organization which aim's to create a fully open source Linux capable RISC-V[2] based SoC. I will be mentored by Stefan Wallentowitz and also co-mentored by Hesham Almatary.

RISC-V is an open source instruction set architecture (ISA) which allows anybody to design, manufacture and sell chips. It was originated in 2010 in the Computer Science division of UC Berkeley.

A Trusted Execution Environment(TEE) allows for isolated execution of code and provides integrity of trusted applications and their assets. ARM has been a pioneer in this field and OPTEE is one such open source solution for the same. Having worked with OPTEE for a good portion of the last six months made me realize the importance of a TEE for any architecture.

I will be looking at OPTEE as a reference and trying to port it to RISC-V. While coming up with the proposal with Stefan, there were three methods to port which were put forward.

1. Direct port of OPTEE: In this method the entire OPTEE-OS code base would have to be ported to RISC-V and we would have to have a software layer that emulates ARM-TrustZone layer . This method would be on the more complex side, I have gone through the code base and have identified parts of OPTEE-OS and libraries which are architecture independent.


2. Running OPTEE on top of seL4 by paravirtualization :
Hesham Almatery has successfully ported seL4 for RISC-V as a part of last year's GSOC , So I could use that.

  • Here seL4 works as a hypervisor.
  • The TrustZone features and secure monitor must be implemented as a seL4 library.
  • OPTEE is paravirtualized, all calls referencing ARM TF and secure monitor are replaced with new calls.
  • I still do not know how virtualization works in seL4, setting up hypervisor(vmm). This is something I have to figure out before the coding period starts.

3.Use seL4 as the Trusted OS: If I go for this approach , libraries will be built for seL4 for it to be used as a secure world OS(TEE).

1. Secure monitor could be built using seL4 IPC mechanism.
2. The normal world OS will be virtualized on top of seL4.
3. Secure boot is readily available for seL4
4. This approach is potentially portable to any platform.
5. seL4 is the secure OS hence we will have to implement secure
storage as well as crypto API’s as seL4 services.
6. A trusted application could be like containers inside seL4.
7. library for crypto libucrypt are architecture independant for the most part hence can be used directly.

It is likely that I will choose to go for the second or third approach as the first approach seems very complex.

[1] -
[2] -

Wednesday, 10 February 2016

How to set up and run OPTEE on QEMU

This is a tutorial on how to run Linaro's OPTEE Trusted Execution Environment on QEMU. I decided to write this post to help someone who might have trouble getting and building the correct versions. I had some trouble initially as well but with the help of Joakim Bech I was successful in building the right version and run my first secure application .Here are some pre-requisites.

1. Linux based machine.
2. Good internet plan
3. Some patience

Some Required Packages:

$ sudo apt-get install android-tools-fastboot autoconf bison cscope curl \ flex gdisk libc6:i386 libfdt-dev libglib2.0-dev \ libpixman-1-dev libstdc++6:i386 libz1:i386 netcat \ python-crypto python-serial uuid-dev xz-utils zlib1g-dev

$ sudo apt-get install ccache

We need to install the repo tool which is used by Android as well to make a central repo of the dependencies.
$ mkdir ~/bin
$ PATH=~/bin:$PATH
$ curl > ~/bin/repo
$ chmod a+x ~/bin/repo

Initialize and sync the repo
$ mkdir -p $HOME/devel/optee
$ cd $HOME/devel/optee
$ repo init -u -m default_stable.xml -b master
$ repo sync
This will take some time to finish depending on your internet speed, so sit back with a cup of coffee. Once this finishes we need to build the toolchains, I had some issues here so I will write my solution as well.
$ cd $HOME/devel/optee/build
$ make toolchains
If this step fails. For some reason it was unable to download the complete files for me so I will post the links where you can manually download the toolchains and paste them in the $HOME/devel/optee/toolchains folder, and I will also post my updated make file. Replace the existing from the build folder.
Updated make file:

Now, time to build!
$ cd $HOME/devel/optee/build
$ make all run

Congratulations, you have successfully built OPTEE, now you can start building applications for it, I will cover that in the next tutorial.

Wednesday, 20 January 2016

Chapter 2 : Looking for Vulnerabilities

A new feature was added to the app which records the user's score and saves the file. Now the quiz display's the user's best score for a particular quiz in the quiz selection menu.

I also had to root my phone to test for some vulnerabilities with the current application.Speaking of the obvious vulnerabilities. There are two basic requirements for this app from the security point of view.
  1. App Integrity: When the app is accessed offline the an other app/user should not be able to change or view the data of the app e.g. The user should not be able to view the questions and answers for the quiz.
  2. Device Integrity: User's progress should not be transferable from one device to another. What would happen is one person complete his quiz and shares his files with the other students.
From the perspective of a non-rooted device both the above requirements are met thanks to Android's Access Control Mechanism which treats each app as a separate user , thus an applications data is actually private. But a rooted user or a malicious application do not follow the same rules.

As we know all the application private data is stored in the data/data directory under root, We could use a simple root file explorer(ES File Explorer) available in the Android market to make the changes we require.

Shown below , a rooted user accessing the questions and answers.

The user can also change his scores.

Tuesday, 5 January 2016

Chapter 1 : The Beginning

This week marks the dawn of the 8th semester, the final semester in my course. We are required to do a project worth 18 credits spanning the entire duration of the semester. I've always wanted to do a project which involves digging down into the hardware level and getting my hands dirty with assembly/low level code, maybe excessive web programming in the past made me lust for a system's project. Back in my 3rd semester the course that changed my outlook towards Computer Science was the Computer Architecture course handled by Dr. Viraj Kumar. We learnt a lot in that course and by far it was my favorite course in the whole of Engineering. So I contacted Professor Viraj along with a couple of my friends(Sreya & Abhinav) and asked him for a project in the system's field. He got back to us in no time and introduced us to a contact of his who works at Microsoft Research, Dr Sriram Rajamani. In a week's time we met to discuss a potential project.

The theme of the project is Secure Computing in Android/ARM devices. Information Security in the traditional sense involves an array of crypto algorithms and secure communication over client/server protocol . In the current landscape if we want to make an app secure so that it cannot be tampered by the user or  malicious software one such solution is to have all the sensitive information stored on the server, this method has its advantages but the main gripe is the application needs to be connected to the internet all the time. This has become a mainstay in the PC gaming industry to curb piracy, recently even Android applications have taken this route to achieve a secure environment. The other problem with this approach is the requirement for an application server which has to cater to all the user's queries. Dr Sriram who has pioneered in the field of Secure Computation interfaces for products such as the Azure suggested that we look into a concept known as enclave. An enclave is a part of application software/code which is isolated from the privileged software (i.e Operating System). As it turns out ARM processors the most popular choice for Android phones has a new feature known as TrustZone which is actually present in 96% of all Android Phones, TrustZone allows for a Trusted Execution Environment(TEE). From our understanding TrustZone is barely used in Android, and most of the solutions are proprietary. So we embarked to build an application which uses TrustZone for secure computing and allow for an offline solution to the classic problem.

Our choice of application was a quiz application. Traditionally questions are retrieved from the server every time and the user's answer is sent to the server and is checked with the answer key and a response is retrieved and the user's score is also maintained in the server. Our application downloads the questions and answers from the server in an encrypted format and the key is stored in the enclave. The quizzes are downloaded only when Internet Connection is available and the result is also stored in the enclave so that the user will not be able to access the answers or modify his score. The user's score will be uploaded to the server when the network is available.

We built a simple quiz application which at the moment is not secure, but works in an offline environment. In this approach we need a remote json file to hold the index for the list of quizzes, the json file will further have links to other remote files containing the quizzes.

We hope to continue our progress and to find a way to enable TrustZone on the Android emulator or QEMU by next week.