Scala on Android

• YouTube
• Twitter
• Medium

• A video from the conference talk
• The slides
• The GitHub repo with examples

This is the first article from a short series I want to make about writing Scala on Android. This one is a transcription of a talk I gave at ScalaLove In The City in February 2021. I think it serves well as an introduction. For four years now I develop the Android client of Wire, an end-to-end encrypted messenger, written in Scala. In Android we deal a lot with events coming from many sources: the user, the backend, the Android OS itself. The code we write has to be very reactive — and it should also be concise and able to process all those events concurrently to squeeze all we can from limited resources. Scala should thrive under those conditions. And yet, it’s almost non-existent. People who still write it are forced to use old versions of libraries, on top of an old version of the language itself, to modify Gradle scripts, and basically to jump through countless loopholes which shouldn’t exist. Most of those people already either moved to other market niches… or to other programming languages. I want to outline how we ended up in this weird position. What attempts were made in the past to introduce Scala on Android, and how they failed. What hacks and concessions are needed in the present to still be able to write Scala on Android. But also I want to tell you about recent developments that give me a reason to believe that the future might be better.

At the I/O 2019 conference, Google announced that the number of Android devices in the world surpassed 2.5 billion. Whatever we think about Google, this is a remarkable achievement. In May 2017 it was 2 billion devices. In September 2015 it was 1.4 billion users… which is not the same as the number of devices, but anyway it shows that the growth is steady even though there’s only a finite number of homo sapiens in the world. In 2019 there were 2.71 billion smartphone users, so even taking into account that some people have more than one device, it clearly shows that Android is by far the biggest platform on mobile phones, and also in absolute numbers, the most popular operating system on Earth. It scores consistently at 70-74% of the mobile market, while its biggest competitor, Apple, is at 25-28%, and other platforms don’t exist. On top of that, the landscape of mobile platforms is uneven. iOS phones are on average more expensive. They are more popular in wealthy countries and among wealthy people — like us. That’s why we may think that Android and iOS are more even than they are. But everywhere around the globe, outside the so-called West, Android’s presence is much much bigger. China, India, Africa, Latin America, Middle East, and even Eastern Europe — those markets are full of Android devices and those markets are growing fast. (Just look here).

But this talk is not about bashing Apple and cheering for Google. Quite the opposite. Here I just want you to see the size of the Android platform because no matter what we think about it, it’s just too big to be ignored.

Android eating apple (https://www.android-apk.com/en/2011/05/android-eating-apple/)

Some more data:

• In 2019 70-80% of internet traffic came from mobile devices.
• A vast majority of it comes from apps, not from mobile-friendly websites, and even though we’re talking mostly about big apps here, Facebook, Twitter, Instagram, etc., those big apps created a culture of looking for an app that will let us do certain things on our phone instead of going to an all-purpose web browser.
• (by the way, a web browser on a mobile device is also an app)
• There are currently around 3 million apps in the Google Play Store. Those apps are written mostly in Java and Kotlin, but also in a few other languages

The sheer number of repositories on GitHub doesn’t mean much without context — but we can look into their proportions: https://github.com/topics/android. Java rules on the Android platform (as of February 16, 2021, there are 43699 repositories, out of 77754, targeting Android with the main programming language being Java), still safely above Kotlin (14438), but we have to remember that Kotlin is a pretty new player — only at the I/O 2019 Google announced that it chose Kotlin as the successor of Java on the Android platform. Javascript is in 3rd place (3315) and Dart is in 4th (2915) because of Flutter which also seems to be the most starred Android project on GitHub and that’s why it’s visible in the background on this screenshot. We will talk about both later too.

But before that… If Java and Kotlin are so prominent, then why don’t we see Scala on this list? After all, it’s the second or third, ex-equo with Kotlin nowadays, language on the JVM. If you dig deeper, you will find that there are 45 repositories where the main programming language is Scala, most of them abandoned or being just small weekend projects. From those that stand out a little, there is the Android client app of my employer, Wire — and there are our friends from 47deg who experimented with Scala on Android a few years ago and even created an app in Scala for keeping track of talks at the Scala Days conferences.

It wasn’t always like this. From 2014 to around 2016 we saw the rise of a few Android projects written in Scala, both libraries and end-user apps. The interest in Android was always small among Scala developers, but for a moment it looked like it can be enough to carve out a small niche for us. After all, Android apps depend on good event handling and reactivity, they are usually small in comparison to backend programs, they are built by small teams that need to act quickly and write safe and sound, but also concise code. Android looked like a perfect place for us. That’s exactly why Wire chose Scala for its Android app. And for some time it made sense. We had three main ways to build Android apps. Two of them through plugins to two main build tools: SBT and Gradle. If you wanted to go the Scala way, there was the sbt-android plugin. If you wanted a more standard Android way, possibly because you wanted to mix your Scala code with Java, you could have used the gradle-scala-android plugin. And there was Scaloid, a library that tried to wrap the hot mess of mutability that is Android SDK into something more suited for Functional Programming.

All three were created in around 2014, were developed and maintained through 2016, and then slowly died in 2017. There are at least a few reasons why it all didn’t work out. There were some decisions made at Google. There was the fact that the Scala community is small and mostly oriented at the backend, and so decisions were made to focus on that… Here, though, I want to talk about the technical problem that for some time proved to be impossible to overcome.

In March 2014, Java 8 was released. Among other features, it introduced support for lambda expressions in Java and the JVM bytecode. Scala, of course, had lambdas much earlier than that, but during the compilation to the bytecode, they had to be replaced with what — on the Java language level — amounted to creating a new inner class in the place where the lambda was used, a method in that class that would be equivalent to the lambda code, then creating an instance of that class, and finally calling the method on that instance. (An example: before, and after). Think for a moment how many lambda expressions you write in your Scala projects all the time, and imagine that for each one of them there would have to be a separate class. You can easily see that it both made the resulting bytecode larger than necessary, and in the runtime, the app used more memory and also was a bit slower because it had to create all those instances. So it was only natural that Scala embraced the new functionality and when in November 2016, version 2.12 was released, it required the new JVM. The resulting bytecode was now smaller and quicker. Developers of Scala libraries and frameworks quickly followed this suite. New versions of their projects took advantage of new lambdas on JVM.

The problem is that what the Android platform uses is not JVM. The old Android’s virtual machine called Dalvik, and the later ART which replaced Dalvik, both take Java bytecode classes and translate them even further into their own bytecode, more optimized for low memory requirements. Dalvik and ART are not required to understand every JVM instruction. And, for example, when Java 8 introduced support for lambda expressions, they stayed as they were. And when Scala 2.12 was released, it turned out it was impossible to use it on Android.

In the beginning, it wasn’t such a big deal. After all, Scala 2.11 was still supported. From the comments on the SBT Android plugin GitHub repo, and its Gitter, we can see that people believed that the lambda problem is just a bug like others. On the Gradle plugin side, we at Wire, and many other people too, forked the plugin and tried some experiments, but without much success. And then the I/O conference happened in May 2017 and Google chose that it will support Kotlin as the main programming language on the Android platform. Together, the Google decision about Kotlin, and the Scala decision about Java 8, so close in time to each other, sent a very strong message. If you prefer Scala, don’t think about programming on Android. If you prefer Android, look into other programming languages, namely, Kotlin. 

Gradually, main libraries of the Scala ecosystem started dropping support for Scala 2.11 and in 2019 even Wire changed its course — we started to look for Kotlin developers and started to rewrite the Wire Android app in Kotlin which slowly continues to this day.

Okay. Maybe it’s Eastern Europe in me speaking. In Eastern Europe we often expect the worst to happen and so then, when it happens, we are prepared for it, get a bit cocky, and say something silly, like “This is problem? This is not problem. When it was -30 and polar bears walked the streets of my town, that was problem…”. We make a show and make sure you know why everything’s wrong, but in the end we go back to work, and get things done. So, I have already told you why everything is wrong. Now let’s get to the second part.

We’re now in 2021 and we’re back at square one. Scala on Android is almost non-existent. We gave up the biggest IT market on the planet without a fight. Do I want to convince you that it makes sense to try again? Yes. That’s exactly what I want. Is it a realistic idea? Well… yes. I wouldn’t be here if I didn’t believe so. But we need to face the facts — no company will risk using Scala on Android in the current situation, both for technical reasons and because there are not enough Scala developers willing to work on Android with old Scala 2.11, and old tools. But how can we have better tools and more Scala developers on Android if companies are not interested in this? It’s a chicken and egg problem that we should have avoided years ago. Now we need another way, through open-source, to overcome the lambda expressions issue, update the tools, and make a small but alive community of people willing to write Scala on Android. Fortunately, we are not alone.

But first, let’s take one thing out of the way. In August 2017 Android 8.0 came out with support for lambda expressions. From this moment, in theory, it’s again possible to update one of the plugins and compile an Android app with Scala 2.12. It wasn’t very practical back then, because in 2017 only a small percent of Android devices had Android 8, but now, in 2021, only around 10% of mobile phones run on Android below 8.0. We could decide that, okay, we will ignore older Androids, and the new plugin will allow for modern Scala on newer devices. Problem solved.

Unfortunately, that’s not true. Even on Androids 8 and up, ART is not completely compatible with Java 8. The lambda issue was the first and the most prominent one, but there are more. We just didn’t get hit by them, because we were stopped by the first one, like, a local boss on the way, in a video game, long before the hero can reach the final level.

This comment comes from the GitHub repo of one of the plugins. The author tried to do exactly what I described above. There can be more problems like that.

In part it’s because Google breaks things all the time. New libraries replace the old ones every year or two, just to be replaced by even newer ones. Important functionalities are getting deprecated and new alternatives are introduced which don’t improve anything much, they just require us to do the same thing differently. Every time a new Android version comes out, the apps that worked fine on the older one start to crash on the new one in random places and Stack Overflow fills up with questions about it and answers like „ooh, since this version you need to do X before you do Y or you will get WTFException in the Z place”. People who worked on the plugins and the Scaloid library just couldn’t possibly catch up. Any smaller company would quickly lose customers if it acted as Google does, but Google is Google, so, where Android developers are supposed to go? We just move along and try to keep sane.

Fortunately, there are few who resist. 

(I know it would make more sense if „Google” and the Android logo switched places on this picture, but I couldn’t resist changing „Gaule” into „Google” like this).

As I mentioned above, there is a small but not insignificant number of them written in Javascript and Dart. They are the result of the fact that Android is not the only end-user platform in existence. Even if it’s the biggest one. There is iOS, but there is also the web, and — believe it or not — there is the desktop. To cover all bases, IT companies need to provide apps for all of them or at least the three major ones. But Android means Java or Kotlin, iOS means Swift, and Web means Javascript in its many forms. That means three teams of developers coding basically the same app but in very different technologies. In response to this situation, a few frameworks were created that try to make it possible to write the code once, and then somehow run it on all platforms and get a similar, decent user experience. We can safely forget about Flutter here — it’s from Google and it requires people to code in Dart — but there are others.

Back in May 2013, Facebook released React, its front-end JavaScript library and in March 2015 it was followed by React Native. Before that Facebook experimented with HTML5 to create a mobile app that would work like a webpage in a container. That attempt failed, but the general idea proved to be interesting… only the implementation had to be on a different level. So, JavaScript, instead of HTML5. React Native allows developers to write their apps in JavaScript and then compile them to native apps on a given platform: Android, but also iOS, Mac, and Windows. If you ever used the Facebook app on your phone, you used React Native. Or if you used Instagram or Discord, or Pinterest, or a bunch of others.

Well, if something can be written in JavaScript, it can also be written in Scala.js. And as a bridge between Scala.js and React Native, we can use Slinky. So here’s the whole stack:

Even though React Native was created by Facebook, it’s an open-source project. You can think of it as an ally in the fight against whatever Google throws at us with every new SDK. It’s not a completely bulletproof shield, but since React Native is open source, and so are Slinky and Scala.js, our relationship with them is on a completely different level than it is with Google. A disadvantage is that… I don’t think people at React Native know much about Scala.js and Slinky. Working with this stack we will surely run into some bugs and things will start to be difficult the moment we will try anything unusual. But, just start experimenting with it, build your app, and if anything goes wrong, go to one of those projects, and file an issue… or even better, try to fix it yourself, with the help of the project’s maintainers. This way the whole community will profit, and the more people will work like that, the better every one of us will be.

Yet another advantage is that by using this stack we write apps not only for Android but at the same time for Web, for iOS, and desktop. There are differences among those platforms and you will need to put some additional effort to have your project work on all of them, but still, it is much better than writing an app directly in Android SDK where, on top of all the other issues, we have that we write only for Android, and If we want the same app to work on an iPhone, we have to write it again from scratch.

So, that’s React Native. But that’s not all. There is at least one more approach I’d like to tell you about, one that gives us a similar set of advantages, but in a different way.

GraalVM is a new Java virtual machine and development kit. It supports more programming languages than you will ever need, and also execution modes, like an ahead-of-time compilation of Java applications for fast startup and low memory footprint. The first production-ready version was released in May 2019, and the current one is from this January. Scala is just one of many possibilities, and in fact, it’s one that was pretty easy to include, as it’s already on JVM. But it’s the other end of GraalVM that is interesting to us — namely, what GraalVM can compile our code to.

From November 2019 GraalVM comes with a tool called Native Image which takes the Java bytecode and turns it into an executable on a given target platform. That platform can be Mac, Linux, Windows, iOS, Android, or even embedded devices. But the ability to compile a program is not enough. An app needs to be able to talk to the underlying operating system. It needs to display widgets on the device’s screen, and the user needs to be able to interact with them. The app also needs access to the storage, to the database if there is one, like on Android, to various functionalities like the audio/video system, and of course, it needs to be able to use the internet connection. In the React Native stack, this is handled by React Native. But GraalVM is just a compiler. We need one more element in this puzzle. That puzzle piece is Gluon, a company that provides functionality for talking to the operating system — be it Android, iOS, or any desktop. Gluon is also the maintainer of JavaFX, a Java UI library. 

JavaFX is not a new thing. It was initially released in 2008 as a part of the standard JDK, and from there it had a bumpy road for some years. In 2018 it became an independent, open-source project, and in my personal opinion since then it’s growing faster than ever and became a base for a few other graphics and widget libraries. Among other things, it provides the UI components for mobile applications written with Gluon and GraalVM.  Or, if you want, on top of JavaFX you can use ScalaFX, but you can also simply use Scala-to-Java converters.

Just as in the previous case, we can write almost the same code for many platforms, and we have a layer shielding us from Android and that layer is open source. If anything goes wrong we can go to one of those projects and try to fix the problem with help from the maintainers. And the disadvantage is, again, that there will be some things going wrong on the borders between all these projects. But also every time someone fixes something or finds a solution, or a workaround, that solution is useful to everyone else. It’s scalable. The more hands we have on board, the better it will get.

A few words about a comparison between those two stacks: In my personal opinion, the difference is mainly that in Scala.js we have all the good stuff that web development gives us. It could be, possibly, a better choice if want a great unconventional user interface. In JavaFX, we might find that it’s a bit more difficult to experiment with UI. On the other hand, the app compiled with GraalVM might be, possibly, smaller and faster, so it could be better if you wanted to write a more resource-heavy app, like a mobile video game, or you want the app to work well on weaker phones   

But I don’t want you to think there’s some sort of competition here. There’s nothing to compete about. At this moment there is almost no Scala presence on Android. No company except Wire writes Scala on Android on a big scale. If right now you will put in your CV that you wrote an Android app in Scala, that won’t give you any points, except maybe as something interesting you can talk about in an interview for a different job. If we want to have Scala on Android, we need to invest our time and energy. We need to reach out to each other and help each other, across time zones, programming languages, and human languages as well. It won’t be easy. But if we do, I can assure you it will be rewarding. It will be fun. And one day we will have our own space on the Android platform, albeit a small one, and we will be able to say “yes, I made it happen”.

One more correction is needed here. I started talking about Scala on Android, but, as I mentioned many times, both approaches I propose are cross-platform. They enable us not only to write Scala on Android. They enable us to write Scala on Everything.

And with this, I think it’s time to wrap up. My project for the year 2021 is to create a bunch of tutorials and examples for the GraalVM tech stack, finally — I hope — leading to me publishing a small mobile video game. Already under this link, you can learn how to build an example Android app: https://github.com/makingthematrix/scalaonandroid
More information will follow in the next months.

Besides, you can always reach out and I’ll try to help the best I can. 

Thanks for reading through all this 😊

Next: How to build an Android app in Scala with GraalVM and JavaFX

Links

Mobile market statistics

• https://gs.statcounter.com/os-market-share/mobile/worldwide
• https://www.statista.com/statistics/266210/number-of-available-applications-in-the-google-play-store/
• https://gs.statcounter.com/android-version-market-share/mobile/worldwide/#monthly-201708-202101
• https://www.reddit.com/r/Android/comments/4u8boi/smartphone_cpu_performance_vs_x86_cpus_are/
• https://expandedramblings.com/index.php/android-statistics/
• https://www.phonearena.com/news/best-high-graphics-games-offline-android-ios_id123524
• https://review42.com/smartphone-statistics/
• https://www.bluecorona.com/blog/mobile-marketing-statistics/
• https://www.ciodive.com/news/70-of-internet-traffic-comes-from-mobile-phones/510120/
• https://www.statista.com/topics/876/android/

I/O Google conferences

• https://events.google.com/io2019/recap
• https://android-developers.googleblog.com/2017/05/google-io-2017-empowering-developers-to.html

Old ways to write Scala on Android

• https://scala-android.org
• https://github.com/saturday06/gradle-android-scala-plugin
• https://github.com/pocorall/scaloid
• https://github.com/scala-android/sbt-android/issues/334#issuecomment-533032010
• https://scalac.io/android-project-with-scala-reflections-on-starting/

Projects written the old way

• https://github.com/wireapp/wire-android/
• https://github.com/47degrees/nine-cards-v2
• https://github.com/47degrees/scala-days-android
• https://github.com/PkmX/lcamera

Lambda expressions issue

• https://www.infoq.com/articles/Scala-2-12-Only-Java8/
• https://stackoverflow.com/questions/14818348/will-android-java-support-lambda-expression-in-java-8
• https://jakewharton.com/androids-java-8-support/
• https://developer.android.com/studio/write/java8-support
• https://www.infoq.com/articles/Java-8-Lambdas-A-Peek-Under-the-Hood/
• https://www.scala-lang.org/news/2.12.0
• https://android.jlelse.eu/closer-look-at-android-runtime-dvm-vs-art-1dc5240c3924

Scala on Android via React Native + Scala.js

• https://reactnative.dev/
• https://slinky.dev/
• https://slinky.dev/docs/scalajs-react-interop/
• https://github.com/japgolly/scalajs-react
• https://engineering.fb.com/2015/03/26/android/react-native-bringing-modern-web-techniques-to-mobile/
• https://www.cleevio.com/blog/16-top-apps-built-with-react-native-in-2020
• https://www.scala-js.org/
• https://www.reddit.com/r/scala/comments/fjtcnf/is_anyone_writing_react_native_with_scala/

Other ways to write Android apps with Javascript 

• Ionic: https://ionicframework.com/
• Capacitor: https://medium.com/geekculture/cross-platform-mobile-dev-with-scala-and-capacitor-54e69b62b50c

Scala on Android via GraalVM + JavaFX

• https://www.graalvm.org/reference-manual/native-image/
• https://docs.gluonhq.com/
• https://gluonhq.com/gluon-substrate-and-graalvm-native-image-with-javafx-support/
• http://www.scalafx.org/
• https://openjfx.io/
• https://github.com/controlsfx/controlsfx

Mobile games frameworks

• https://github.com/AlmasB/FXGL
• https://libgdx.badlogicgames.com/
• https://www.youtube.com/watch?v=qlJUrcpQXo8

Om nom nom: https://www.android-apk.com/en/2011/05/android-eating-apple/

Reklama