The Java Class Library (JCL) – with Java being one of the majorly adopted programming languages – is heavily used and an implicitly trusted library on which many mission critical applications are based. In order to prevent abuse, Java has a sophisticated security model to ensure the isolation of protected areas inside a program. However, attackers have found and continue to find several ways to disable the security model thus rendering it useless.
One way of effectively evading the Java Security Model is to perform operations in native code. Since attackers cannot easily introduce new native libraries during an attack, they are keen to abuse an exploitable part of the native code already provided by the JCL itself. As this is not a small part (roughly 800k LOC in Java 1.7) of the JCL, a manual code review looking for security vulnerabilities is hardly an option. Automated methods have to be developed to mitigate the possible threat the native part of the JCL poses.
When constructing an attack against the Java Security Model using the native part of the JCL most attacks use specially crafted input sent through Java methods to the native part. This crafted input might break the native part and thus enable the Java part of the exploit to deactivate the Java Security Model (e.g. CVE-2013-2465) and continue in full privileged mode. Choosing an Applet as the delivery method for the exploit the number of possible targets easily becomes interesting for an attacker.
In this thesis an automated analysis of the data flows between the VM-controlled and the native part of the JCL has to be created. As it will be hard to cross the language boundary between the VM-controlled and the native part with an analysis, the analysis may run in two steps. One step analyzing the Java part of the JCL (e.g. with Soot) and another step analyzing the native part of the JCL (e.g. with LLVM). The results of both analysis steps then have to be combined to produce an overall result. A classification schema has to be developed to characterize data flows depending on their possible exploitability. For instance, some safe guards and input sanitizers might mitigate threats well, while others might not. Additionally, certain data types could be more prone to exploitation than other data types. However, some parameters of the Java Native API might not even be accessible for an attacker at all.
You will find the original thesis description on the institute’s website. If you are a CS student of TU Darmstadt, please contact me if you are interested in making this topic the topic of your Bachelor or Master’s thesis.