[ad_1]
Posted by Joel Galenson and Matthew Maurer, Android Staff
One of many major challenges of evaluating Rust to be used inside the Android platform was guaranteeing we may present ample interoperability with our current codebase. If Rust is to fulfill its targets of bettering safety, stability, and high quality Android-wide, we’d like to have the ability to use Rust wherever within the codebase that native code is required. To perform this, we have to present nearly all of performance platform builders use. As we mentioned beforehand, now we have an excessive amount of C++ to contemplate ignoring it, rewriting all of it’s infeasible, and rewriting older code would doubtless be counterproductive because the bugs in that code have largely been fastened. This implies interoperability is essentially the most sensible method ahead.
Earlier than introducing Rust into the Android Open Supply Challenge (AOSP), we wanted to show that Rust interoperability with C and C++ is ample for sensible, handy, and protected use inside Android. Including a brand new language has prices; we wanted to show that Rust would have the ability to scale throughout the codebase and meet its potential so as to justify these prices. This submit will cowl the evaluation we did greater than a 12 months in the past whereas we evaluated Rust to be used in Android. We additionally current a follow-up evaluation with some insights into how the unique evaluation has held up as Android initiatives have adopted Rust.
Present language interoperability in Android focuses on properly outlined foreign-function interface (FFI) boundaries, which is the place code written in a single programming language calls into code written in a distinct language. Rust assist will likewise deal with the FFI boundary as that is in keeping with how AOSP initiatives are developed, how code is shared, and the way dependencies are managed. For Rust interoperability with C, the C utility binary interface (ABI) is already ample.
Interoperability with C++ is more difficult and is the main target of this submit. Whereas each Rust and C++ assist utilizing the C ABI, it isn’t ample for idiomatic utilization of both language. Merely enumerating the options of every language leads to an unsurprising conclusion: many ideas will not be simply translatable, nor can we essentially need them to be. In any case, we’re introducing Rust as a result of many options and traits of C++ make it troublesome to jot down protected and proper code. Due to this fact, our objective is to not take into account all language options, however reasonably to research how Android makes use of C++ and make sure that interop is handy for the overwhelming majority of our use circumstances.
We analyzed code and interfaces within the Android platform particularly, not codebases on the whole. Whereas this implies our particular conclusions might not be correct for different codebases, we hope the methodology will help others to make a extra knowledgeable resolution about introducing Rust into their massive codebase. Our colleagues on the Chrome browser workforce have performed an identical evaluation, which you’ll find right here.
This evaluation was not initially supposed to be revealed outdoors of Google: our objective was to make a data-driven resolution on whether or not or not Rust was a sensible choice for methods improvement in Android. Whereas the evaluation is meant to be correct and actionable, it was by no means supposed to be complete, and we’ve identified a few areas the place it may very well be extra full. Nevertheless, we additionally observe that preliminary investigations into these areas confirmed that they might not considerably affect the outcomes, which is why we determined to not make investments the extra effort.
Exported features from Rust and C++ libraries are the place we take into account interop to be important. Our targets are easy:
Rust should have the ability to name features from C++ libraries and vice versa.
FFI ought to require a minimal of boilerplate.
FFI mustn’t require deep experience.
Whereas making Rust features callable from C++ is a objective, this evaluation focuses on making C++ features obtainable to Rust in order that new Rust code will be added whereas profiting from current implementations in C++. To that finish, we take a look at exported C++ features and take into account current and deliberate compatibility with Rust by way of the C ABI and compatibility libraries. Sorts are extracted by working objdump on shared libraries to seek out exterior C++ features they use1 and working c++filt to parse the C++ sorts. This offers features and their arguments. It doesn’t take into account return values, however a preliminary analysis2 of these revealed that they might not considerably have an effect on the outcomes.
We then classify every of those sorts into one of many following buckets:
Supported by bindgen
These are usually easy sorts involving primitives (together with pointers and references to them). For these sorts, Rust’s current FFI will deal with them accurately, and Android’s construct system will auto-generate the bindings.
Supported by cxx compat crate
These are dealt with by the cxx crate. This at the moment consists of std::string, std::vector, and C++ strategies (together with pointers/references to those sorts). Customers merely need to outline the kinds and features they need to share throughout languages and cxx will generate the code to try this safely.
Native assist
These sorts will not be immediately supported, however the interfaces that use them have been manually reworked so as to add Rust assist. Particularly, this consists of sorts utilized by AIDL and protobufs.
Now we have additionally applied a local interface for StatsD as the prevailing C++ interface depends on methodology overloading, which isn’t properly supported by bindgen and cxx3. Utilization of this method doesn’t present up within the evaluation as a result of the C++ API doesn’t use any distinctive sorts.
Potential addition to cxx
That is at the moment widespread information buildings similar to std::optionally available and std::chrono::period and customized string and vector implementations.
These can both be supported natively by a future contribution to cxx, or through the use of its ExternType amenities. Now we have solely included sorts on this class that we imagine are comparatively simple to implement and have an inexpensive likelihood of being accepted into the cxx undertaking.
We do not want/intend to assist
Some sorts are uncovered in right this moment’s C++ APIs which are both an implicit a part of the API, not an API we count on to need to use from Rust, or are language particular. Examples of sorts we don’t intend to assist embrace:
Mutexes – we count on that locking will happen in a single language or the opposite, reasonably than needing to move mutexes between languages, as per our coarse-grained philosophy.
native_handle – this can be a JNI interface sort, so it’s inappropriate to be used in Rust/C++ communication.
std::locale& – Android makes use of a separate locale system from C++ locales. This sort primarily seems in output resulting from e.g., cout utilization, which might be inappropriate to make use of in Rust.
General, this class represents sorts that we don’t imagine a Rust developer must be utilizing.
HIDL
Android is within the technique of deprecating HIDL and migrating to AIDL for HALs for brand spanking new companies.We’re additionally migrating some current implementations to steady AIDL. Our present plan is to not assist HIDL, preferring emigrate to steady AIDL as an alternative. These sorts thus at the moment fall into the “We do not want/intend to assist” bucket above, however we break them out to be extra particular. If there may be ample demand for HIDL assist, we could revisit this resolution later.
Different
This accommodates all kinds that don’t match into any of the above buckets. It’s at the moment largely std::string being handed by worth, which isn’t supported by cxx.
One of many major causes for supporting interop is to permit reuse of current code. With this in thoughts, we decided essentially the most generally used C++ libraries in Android: liblog, libbase, libutils, libcutils, libhidlbase, libbinder, libhardware, libz, libcrypto, and libui. We then analyzed all the exterior C++ features utilized by these libraries and their arguments to find out how properly they might interoperate with Rust.
General, 81% of sorts are within the first three classes (which we at the moment absolutely assist) and 87% are within the first 4 classes (which incorporates these we imagine we will simply assist). Virtually all the remaining sorts are these we imagine we don’t have to assist.
Along with analyzing widespread C++ libraries, we additionally examined Mainline modules. Supporting this context is vital as Android is migrating a few of its core performance to Mainline, together with a lot of the native code we hope to enhance with Rust. Moreover, their modularity presents a possibility for interop assist.
We analyzed 64 binaries and libraries in 21 modules. For every analyzed library we examined their used C++ features and analyzed the forms of their arguments to find out how properly they might interoperate with Rust in the identical method we did above for the highest 10 libraries.
Right here 88% of sorts are within the first three classes and 90% within the first 4, with virtually all the remaining being sorts we don’t have to deal with.
With virtually a 12 months of Rust improvement in AOSP behind us, and greater than 100 thousand traces of code written in Rust, we will now study how our unique evaluation has held up primarily based on how C/C++ code is at the moment known as from Rust in AOSP.4
The outcomes largely match what we anticipated from our evaluation with bindgen dealing with nearly all of interop wants. In depth use of AIDL by the brand new Keystore2 service leads to the first distinction between our unique evaluation and precise Rust utilization within the “Native Assist” class.Just a few present examples of interop are:
Cxx in Bluetooth – Whereas Rust is meant to be the first language for Bluetooth, migrating from the prevailing C/C++ implementation will occur in phases. Utilizing cxx permits the Bluetooth workforce to extra simply serve legacy protocols like HIDL till they’re phased out through the use of the prevailing C++ assist to incrementally migrate their service.
AIDL in keystore – Keystore implements AIDL companies and interacts with apps and different companies over AIDL. Offering this performance can be troublesome to assist with instruments like cxx or bindgen, however the native AIDL assist is straightforward and ergonomic to make use of.
Manually-written wrappers in profcollectd – Whereas our objective is to supply seamless interop for many use circumstances, we additionally need to show that, even when auto-generated interop options will not be an possibility, manually creating them will be easy and easy. Profcollectd is a small daemon that solely exists on non-production engineering builds. As an alternative of utilizing cxx it makes use of some small manually-written C wrappers round C++ libraries that it then passes to bindgen.
Bindgen and cxx present the overwhelming majority of Rust/C++ interoperability wanted by Android. For a number of the exceptions, similar to AIDL, the native model gives handy interop between Rust and different languages. Manually written wrappers can be utilized to deal with the few remaining sorts and features not supported by different choices in addition to to create ergonomic Rust APIs. General, we imagine interoperability between Rust and C++ is already largely ample for handy use of Rust inside Android.
In case you are contemplating how Rust may combine into your C++ undertaking, we advocate doing an identical evaluation of your codebase. When addressing interop gaps, we advocate that you just take into account upstreaming assist to current compat libraries like cxx.
Our first try at quantifying Rust/C++ interop concerned analyzing the potential mismatches between the languages. This led to plenty of fascinating info, however was troublesome to attract actionable conclusions from. Moderately than enumerating all of the potential locations the place interop may happen, Stephen Hines instructed that we as an alternative take into account how code is at the moment shared between C/C++ initiatives as an inexpensive proxy for the place we’ll additionally doubtless need interop for Rust. This supplied us with actionable info that was simple to prioritize and implement. Wanting again, the information from our real-world Rust utilization has strengthened that the preliminary methodology was sound. Thanks Stephen!Additionally, because of:
Andrei Homescu and Stephen Crane for contributing AIDL assist to AOSP.
Ivan Lozano for contributing protobuf assist to AOSP.
David Tolnay for publishing cxx and accepting our contributions.
The various authors and contributors to bindgen.
Jeff Vander Stoep and Adrian Taylor for contributions to this submit.
[ad_2]