[ad_1]
Posted by Joel Galenson and Matthew Maurer, Android Group
One of many primary challenges of evaluating Rust to be used throughout the Android platform was guaranteeing we may present ample interoperability with our present codebase. If Rust is to fulfill its targets of enhancing safety, stability, and high quality Android-wide, we’d like to have the ability to use Rust wherever within the codebase that native code is required. To perform this, we have to present nearly all of performance platform builders use. As we mentioned beforehand, we’ve an excessive amount of C++ to think about ignoring it, rewriting all of it’s infeasible, and rewriting older code would probably be counterproductive because the bugs in that code have largely been fastened. This implies interoperability is probably the most sensible manner ahead.
Earlier than introducing Rust into the Android Open Supply Mission (AOSP), we would have liked to display that Rust interoperability with C and C++ is ample for sensible, handy, and secure use inside Android. Including a brand new language has prices; we would have liked to display that Rust would have the ability to scale throughout the codebase and meet its potential with the intention to justify these prices. This submit will cowl the evaluation we did greater than a yr in the past whereas we evaluated Rust to be used in Android. We additionally current a follow-up evaluation with some insights into how the unique evaluation has held up as Android tasks have adopted Rust.
Present language interoperability in Android focuses on nicely outlined foreign-function interface (FFI) boundaries, which is the place code written in a single programming language calls into code written in a unique language. Rust assist will likewise deal with the FFI boundary as that is in line with how AOSP tasks are developed, how code is shared, and the way dependencies are managed. For Rust interoperability with C, the C software binary interface (ABI) is already ample.
Interoperability with C++ is more difficult and is the main target of this submit. Whereas each Rust and C++ assist utilizing the C ABI, it isn’t ample for idiomatic utilization of both language. Merely enumerating the options of every language ends in an unsurprising conclusion: many ideas aren’t simply translatable, nor will we essentially need them to be. In any case, we’re introducing Rust as a result of many options and traits of C++ make it tough to jot down secure and proper code. Subsequently, our purpose is to not contemplate all language options, however quite to investigate how Android makes use of C++ and make sure that interop is handy for the overwhelming majority of our use instances.
We analyzed code and interfaces within the Android platform particularly, not codebases usually. Whereas this implies our particular conclusions will not be correct for different codebases, we hope the methodology might help others to make a extra knowledgeable resolution about introducing Rust into their massive codebase. Our colleagues on the Chrome browser group have achieved an analogous evaluation, which you will discover right here.
This evaluation was not initially supposed to be printed exterior of Google: our purpose was to make a data-driven resolution on whether or not or not Rust was a good selection for programs growth in Android. Whereas the evaluation is meant to be correct and actionable, it was by no means supposed to be complete, and we’ve identified a few areas the place it might be extra full. Nonetheless, we additionally word that preliminary investigations into these areas confirmed that they’d not considerably impression the outcomes, which is why we determined to not make investments the extra effort.
Exported features from Rust and C++ libraries are the place we contemplate interop to be important. Our targets are easy:
Rust should have the ability to name features from C++ libraries and vice versa.
FFI ought to require a minimal of boilerplate.
FFI shouldn’t require deep experience.
Whereas making Rust features callable from C++ is a purpose, this evaluation focuses on making C++ features obtainable to Rust in order that new Rust code will be added whereas benefiting from present implementations in C++. To that finish, we have a look at exported C++ features and contemplate present and deliberate compatibility with Rust through the C ABI and compatibility libraries. Varieties are extracted by operating objdump on shared libraries to search out exterior C++ features they use1 and operating c++filt to parse the C++ varieties. This offers features and their arguments. It doesn’t contemplate return values, however a preliminary analysis2 of these revealed that they’d not considerably have an effect on the outcomes.
We then classify every of those varieties into one of many following buckets:
Supported by bindgen
These are usually easy varieties involving primitives (together with pointers and references to them). For these varieties, Rust’s present FFI will deal with them appropriately, and Android’s construct system will auto-generate the bindings.
Supported by cxx compat crate
These are dealt with by the cxx crate. This at the moment contains std::string, std::vector, and C++ strategies (together with pointers/references to those varieties). Customers merely must outline the categories and features they wish to share throughout languages and cxx will generate the code to do this safely.
Native assist
These varieties aren’t immediately supported, however the interfaces that use them have been manually reworked so as to add Rust assist. Particularly, this contains varieties utilized by AIDL and protobufs.
We now have additionally applied a local interface for StatsD as the present C++ interface depends on methodology overloading, which isn’t nicely supported by bindgen and cxx3. Utilization of this method doesn’t present up within the evaluation as a result of the C++ API doesn’t use any distinctive varieties.
Potential addition to cxx
That is at the moment frequent knowledge buildings similar to std::optionally available and std::chrono::period and customized string and vector implementations.
These can both be supported natively by a future contribution to cxx, or by utilizing its ExternType amenities. We now have solely included varieties on this class that we imagine are comparatively easy to implement and have an affordable likelihood of being accepted into the cxx challenge.
We do not want/intend to assist
Some varieties are uncovered in at present’s C++ APIs which are both an implicit a part of the API, not an API we count on to wish to use from Rust, or are language particular. Examples of varieties we don’t intend to assist embody:
Mutexes – we count on that locking will happen in a single language or the opposite, quite than needing to cross mutexes between languages, as per our coarse-grained philosophy.
native_handle – it is a JNI interface sort, so it’s inappropriate to be used in Rust/C++ communication.
std::locale& – Android makes use of a separate locale system from C++ locales. This sort primarily seems in output resulting from e.g., cout utilization, which might be inappropriate to make use of in Rust.
Total, this class represents varieties that we don’t imagine a Rust developer needs to be utilizing.
HIDL
Android is within the means of deprecating HIDL and migrating to AIDL for HALs for brand spanking new providers.We’re additionally migrating some present implementations to secure AIDL. Our present plan is to not assist HIDL, preferring emigrate to secure AIDL as an alternative. These varieties thus at the moment fall into the “We do not want/intend to assist” bucket above, however we break them out to be extra particular. If there’s ample demand for HIDL assist, we might revisit this resolution later.
Different
This incorporates every kind that don’t match into any of the above buckets. It’s at the moment principally std::string being handed by worth, which isn’t supported by cxx.
One of many main causes for supporting interop is to permit reuse of present code. With this in thoughts, we decided probably the most generally used C++ libraries in Android: liblog, libbase, libutils, libcutils, libhidlbase, libbinder, libhardware, libz, libcrypto, and libui. We then analyzed all the exterior C++ features utilized by these libraries and their arguments to find out how nicely they’d interoperate with Rust.
Total, 81% of varieties are within the first three classes (which we at the moment absolutely assist) and 87% are within the first 4 classes (which incorporates these we imagine we are able to simply assist). Nearly all the remaining varieties are these we imagine we don’t have to assist.
Along with analyzing fashionable C++ libraries, we additionally examined Mainline modules. Supporting this context is crucial as Android is migrating a few of its core performance to Mainline, together with a lot of the native code we hope to reinforce with Rust. Moreover, their modularity presents a chance for interop assist.
We analyzed 64 binaries and libraries in 21 modules. For every analyzed library we examined their used C++ features and analyzed the varieties of their arguments to find out how nicely they’d interoperate with Rust in the identical manner we did above for the highest 10 libraries.
Right here 88% of varieties are within the first three classes and 90% within the first 4, with nearly all the remaining being varieties we don’t have to deal with.
With nearly a yr of Rust growth in AOSP behind us, and greater than 100 thousand traces of code written in Rust, we are able to now study how our unique evaluation has held up based mostly on how C/C++ code is at the moment referred to as from Rust in AOSP.4
The outcomes largely match what we anticipated from our evaluation with bindgen dealing with nearly all of interop wants. In depth use of AIDL by the brand new Keystore2 service ends in the first distinction between our unique evaluation and precise Rust utilization within the “Native Assist” class.Just a few present examples of interop are:
Cxx in Bluetooth – Whereas Rust is meant to be the first language for Bluetooth, migrating from the present C/C++ implementation will occur in levels. Utilizing cxx permits the Bluetooth group to extra simply serve legacy protocols like HIDL till they’re phased out by utilizing the present C++ assist to incrementally migrate their service.
AIDL in keystore – Keystore implements AIDL providers and interacts with apps and different providers over AIDL. Offering this performance can be tough to assist with instruments like cxx or bindgen, however the native AIDL assist is easy and ergonomic to make use of.
Manually-written wrappers in profcollectd – Whereas our purpose is to supply seamless interop for many use instances, we additionally wish to display that, even when auto-generated interop options aren’t an choice, manually creating them will be easy and easy. Profcollectd is a small daemon that solely exists on non-production engineering builds. As an alternative of utilizing cxx it makes use of some small manually-written C wrappers round C++ libraries that it then passes to bindgen.
Bindgen and cxx present the overwhelming majority of Rust/C++ interoperability wanted by Android. For a few of the exceptions, similar to AIDL, the native model gives handy interop between Rust and different languages. Manually written wrappers can be utilized to deal with the few remaining varieties and features not supported by different choices in addition to to create ergonomic Rust APIs. Total, we imagine interoperability between Rust and C++ is already largely ample for handy use of Rust inside Android.
In case you are contemplating how Rust may combine into your C++ challenge, we advocate doing an analogous evaluation of your codebase. When addressing interop gaps, we advocate that you just contemplate upstreaming assist to present compat libraries like cxx.
Our first try at quantifying Rust/C++ interop concerned analyzing the potential mismatches between the languages. This led to numerous attention-grabbing info, however was tough to attract actionable conclusions from. Reasonably than enumerating all of the potential locations the place interop may happen, Stephen Hines prompt that we as an alternative contemplate how code is at the moment shared between C/C++ tasks as an affordable proxy for the place we’ll additionally probably need interop for Rust. This offered us with actionable info that was easy to prioritize and implement. Trying again, the information from our real-world Rust utilization has strengthened that the preliminary methodology was sound. Thanks Stephen!Additionally, because of:
Andrei Homescu and Stephen Crane for contributing AIDL assist to AOSP.
Ivan Lozano for contributing protobuf assist to AOSP.
David Tolnay for publishing cxx and accepting our contributions.
The numerous authors and contributors to bindgen.
Jeff Vander Stoep and Adrian Taylor for contributions to this submit.
[ad_2]