Andreas Kling

Making myself uncomfortable again

2023-04-07T00:00:00+00:00

In early 2019, some months after completing a rehab program for drug addiction, I was in a very open-minded headspace where I wanted to challenge myself and find ways to improve as a person.

Drugs had filled my life with secrecy and lies, but that life was over. Although I was unsure of my next steps, I was so tired of the secrecy that I couldn’t think of anything I’d rather change about myself.

So, roughly 6 months into the SerenityOS project, I started making YouTube videos to document my progress. At the time, I was still working on the project alone, and I thought video could be an engaging format.

The first videos were simple programming sessions, filled with clumsiness and mumbling. I made lots of mistakes and mumbled even more while fixing them. Over time, I improved, and also starting making monthly SerenityOS update videos – a tradition which is still going today.

Eventually, I began recording myself driving to work, and that’s when I truly opened up. In these car videos, I discussed programming, job experiences, motivation, operating systems, and many other topics. As I grew more comfortable, I eventually shared my history with drug addiction and how it inspired the name “SerenityOS” as well. People were incredibly supportive, and for the first time since childhood I felt accepted for who I really was.

Where I am today

Just two weeks ago, I posted my 1000th video to YouTube. That’s a lot of videos. With more than 39,600 subscribers, I’ve been able to share my work, thoughts and knowledge with more people than I ever imagined.

Moreover, I’ve attracted hundreds of people to collaborate on SerenityOS with me, many of whom are still active contributors today. This experience has brought me immense joy.

At this point, I feel like I’ve accomplished my goal of becoming comfortable on camera and learned to embrace authenticity, leaving the secrecy and lies behind.

Now, I’m ready to face a new challenge and feel uncomfortable once again!

Writing: a new frontier

Writing has always intimidated me. While I can manage short bursts (like Tweets), I’ve rarely practiced writing anything more substantial. This is something I’m incredibly uncomfortable with, and I feel like it’s time to confront it.

So, here’s what I’m going to do: I’ll be taking a break from making my usual “unprepared” videos (i.e everything other than SerenityOS update videos), and focus instead on writing. I will write posts like this one, but also technical posts, book reviews, etc.

As I write this, a mix of fear and excitement fills my heart. I feel intimidated and out of my element, and this tells me I’m on the right track.

With this, I aim to become significantly better at writing. I also hope to become better at thinking. I tend to think in English, so it seems logical that improving my command of English will improve my ability to think as well.

To follow along on my new journey, stay tuned for regular blog updates and post announcements on my Twitter.

It’s going to be bumpy at first, but I’ll do my best to improve quickly.

Thank you for reading! I hope to see you here again.

MutexProtected: A C++ Pattern for Easier Concurrency

2023-04-06T00:00:00+00:00

In this post, we will discuss the challenges of programming with locks and how the C++ language offers some useful tools to make it easier. We will start with an example in C and then use C++ to improve upon it in steps. The example APIs are based on real-life APIs from the SerenityOS kernel.

Barebones example in C

Let’s imagine a struct Thing with a field field that will be accessed by multiple threads. We’ll use a mutex mutex to ensure that only one thread can access it at a time.

struct Thing {
    Mutex mutex;
    Field field;
};

In C, accessing Thing would typically look something like this:

mutex_lock(&thing->mutex);
use(&thing->field);
mutex_unlock(&thing->mutex);

An obvious problem in the C version is that the mutex must be manually unlocked. Forgetting to unlock a mutex tends to have unpleasant consequences.

Improving it with a C++ RAII class

The “forgot to unlock” problem is easily solved in C++ with a RAII class for automatic locking & unlocking:

{
    MutexLocker locker(thing->mutex);
    use(thing->field);
}

The MutexLocker locks the mutex when constructed and unlocks it when destroyed. No need for a manual call to mutex_unlock() anymore.

That’s already pretty good! However, the MutexLocker approach still has some major shortcomings:

You can still forget to lock the mutex and access field anyway.
Developers who are unfamiliar with this code may not realize that mutex and field have this important relationship.

Even so, the MutexLocker was our favored pattern in SerenityOS up until 2021, when we introduced a new pattern to the codebase.

Adding lambdas to the mix: introducing `MutexProtected`

MutexProtected is a powerful C++ construct that addresses the main issues with MutexLocker and makes it significantly easier to use mutexes correctly:

struct Thing {
    MutexProtected<Field> field;
};

thing->field.with([&](Field& field) {
    use(field);
});

Essentially, MutexProtected<T> is a bundled mutex and T. However, you can’t access the T directly! The only way we’ll let you access the T is by calling with() and passing it a callback that takes a T& parameter.

When called, with() locks the mutex, then invokes the callback, and finally unlocks the mutex again before returning.

As you can see, we’ve now also solved the issue of someone forgetting to lock the mutex before accessing the field. And not only that, but since the mutex and field have been combined into a single variable, you no longer have to be aware of the relationship between the two. It’s been encoded into the type system!

When multiple fields are protected by a single mutex, we can simply combine them into a struct:

struct ManyFields {
    Field1 field1;
    Field2 field2;
    Field3 field3;
};

struct Thing {
    MutexProtected<ManyFields> fields;
};

Note that it’s still possible to make mistakes with MutexProtected, such as deadlocking the program by using multiple MutexProtected simultaneously in inconsistent order. Thankfully such bugs are generally trivial to diagnose compared to data races.

That concludes our look at MutexProtected. If you’re currently working on a C++ project using mainly the MutexLocker approach, consider adding something like our MutexProtected to further reduce the chances of using locks incorrectly.

You can find the SerenityOS implementation on GitHub. (Note that it’s a little more sophisticated than the imaginary MutexProtected I’ve used for examples above.)

Final notes

Although MutexProtected was introduced to SerenityOS in 2021, we do still have a lot of code using the old MutexLocker pattern. There are still cases where MutexLocker works better, for example when a mutex is used to synchronize something other than data.

And yes, it is possible to make life difficult by persisting the T& to an outside location while inside the callback. This is C++ after all, so the programmer does have great freedom. However, doing so is definitely not recommended, and we have yet to encounter anyone trying to do this in our codebase.

Excellence is a habit, but so is failure

2023-04-05T00:00:00+00:00

We often hear that making small incremental improvements every day can lead to great things. This popular piece of advice rings true, and it’s a powerful reminder to keep pushing ourselves forward.

But there’s another side to this story that we don’t discuss as often: how incremental neglect and small missteps can accumulate and lead to negative outcomes. Recognizing and addressing these patterns of neglect early can make a significant difference in preventing larger problems down the road.

Reflecting on my own life, I’ve noticed that most of the things that really went off-track were indeed consequences of incremental neglect and numerous small yet poor choices:

I didn’t become addicted to drugs overnight. It happened over hundreds of moments where I prioritized momentary pleasure over health and safety.
I didn’t become overweight overnight. It happened over hundreds of moments where I opted for immediate gratification over long-term health.
I didn’t ruin relationships overnight. It happened over hundreds of moments where I chose comfort over confronting difficult conversations, admitting my mistakes, or even just acknowledging that someone was better than me at something.

From these experiences, I’ve realized that avoiding bad habits is just as important as cultivating good habits.

To address these kinds of issues, we must become aware of our patterns of incremental neglect and then take deliberate steps to counteract them and foster healthier habits.

How SerenityOS declares ssize_t

2023-04-04T00:00:00+00:00

This post explores one of my favorite hacks in SerenityOS. I don’t recommend doing this in your codebase, but it has worked for us so far. :^)

Background

size_t and ssize_t are common types used in many POSIX APIs. According to POSIX, they are used as follows:

size_t: Used for sizes of objects.
ssize_t: Used for a count of bytes or an error indication.

In practice, ssize_t is essentially a “signed size_t”.

Since we’re building the whole operating system, including the standard C library ourselves, we’re also responsible for declaring all the common system types, including size_t and ssize_t.

How we declare them

To declare size_t, we leverage the C preprocessor’s predefined __SIZE_TYPE__ macro:

typedef __SIZE_TYPE__ size_t;

However, there is no __SSIZE_TYPE__ macro for ssize_t, so I decided to get a little creative:

#define unsigned signed
typedef __SIZE_TYPE__ ssize_t;
#undef unsigned

Here’s what’s happening: The C preprocessor expands “__SIZE_TYPE__” to “unsigned long” or something similar. We trick it by temporarily defining a macro that replaces “unsigned” with “signed”, and so ssize_t is declared as a signed version of whatever the size_t type is!

How others declare them

Other C libraries typically use more careful techniques, such as wrapping the declarations in architecture-specific #ifdefs:

#ifdef __i386__
typedef uint32_t size_t;
typedef int32_t ssize_t;
#endif

#ifdef __x86_64__
typedef uint64_t size_t;
typedef int64_t ssize_t;
#endif

That’s obviously a better approach if you’re building with compilers that don’t predefine __SIZE_TYPE__, or in an environment where unsigned has already been redefined.

In our case, we haven’t had any issues with our approach yet, so I’m inclined to hold on to the hack as it’s just so cute. :^)

15 Minutes Every Day

2023-04-03T00:00:00+00:00

Someone once suggested that I set aside a small portion of my income every month. I’ve been doing this for years with an automatic bank transfer so I can’t mess it up.

I was reflecting on this habit recently, and had a thought: what if I treated cleaning the same way?

I’ve never been good at cleaning. Every place I ever lived was a bit messy, except when expecting a visitor.

The idea became: every day I’d “invest” 15 minutes into our home by picking a small cleaning task and doing it immediately after breakfast. I shared the idea with my wife, and we agreed to test it out together.

For the first few days, we found things to do easily, such as organizing cables, tools, pantry items and clothes. Then, we moved on to cleaning tasks, like scrubbing cabinets, washing floors, and cleaning the sofa.

These kind of things used to be so hard for me to get done, but now I’m doing them every day, and it’s even fun doing them together with my wife. We also do a little “demo” when we’re finished, to show each other what we accomplished.

We’re only 2 weeks into this, and our home looks noticeably nicer. I also find it easier to keep it nice, since any small mess now stands out way more than it did before.

I’m sure there’s a novelty aspect to why I’m enjoying this, but even so, it really does feel like a sustainable habit, especially since it’s only 15 minutes every day.

How I make a living working on SerenityOS

2022-10-29T00:00:00+00:00

This post describes in detail how I support myself while working on the SerenityOS project.

Introduction

Hello! My name is Andreas Kling and I’m the founder of the SerenityOS project. If you’re not familiar, SerenityOS is a from-scratch operating system that I started building in 2018. It combines a Unix-like core with the look & feel of the late 1990s. Although it began as a one-man project, it has since blossomed into a lively open source community with hundreds of amazing developers working on it.

Since May of 2021, SerenityOS has been my full-time job. This terrifying but exciting leap was made possible by recurring donations from generous people who like what I do and want to see it continue.

With so many people having my back, I’ve grown a bit more courageous with taking on large projects, which has led to SerenityOS expanding into both a new memory-safe systems language (Jakt) and a new cross-platform browser (Ladybird) in the last year.

“Okay, but how do you actually make a living?”

I get this question regularly, so I’m going to do my best to answer it. Please understand that I’m publishing this for transparency, not to brag about making so much or complain about not making enough.

All currency amounts below are in USD.

Main source of income: Individual sponsorships/donations

I created a Patreon back in April of 2019. I felt a bit silly at the time, with thoughts like “who do I think I am” and “what am I even doing” echoing in my head. I still did it though. I was too curious to see what would happen, even though I expected nothing. Amazingly, a couple of people actually signed up!

Later on, when GitHub Sponsors opened up, I got in as early as I could. Not only did they have zero fees, but they also offered to match the first $5,000 in donations. And once again, some people actually signed up!

On both platforms, people are invited to donate to me personally so that I can spend time working on SerenityOS. No goods/services are offered in return.

Here are the current numbers as of 2022-10-29. They tend to fluctuate as people join and leave as supporters.

Platform	Supporters	$/month
GitHub Sponsors	263	2,136
Patreon	268	1,411
PayPal	<5	<200
Cryptocurrency	<5	<50

Early on I would monitor these numbers closely, as I found it fascinating. Eventually it became a source of stress, so now I only check on them once a month when I do my accounting.

Secondary source of income: YouTube

I’ve been fairly active on YouTube since early on in the SerenityOS project. I started uploading development videos about ~6 months into the project. They were awkward and clumsy, but people seemed to enjoy watching someone incrementally building a new operating system from scratch.

Once the channel grew large enough, I was able to enable monetization in the form of ads. I felt a bit weird about this, since I use an ad blocker myself, but I figured that the kind of person who watches my content is perfectly aware of ad blockers and can make their own decisions about them.

Income from ads varies wildly depending on how often I upload new videos, and how many views they get. YouTube’s terms & conditions don’t allow me to disclose analytics in detail, but my estimated revenue this month is $315.

Outside of ads, I also have 58 people enrolled as “channel members”, which nets $65/month (part of the $315). YouTube allows you to create separate content for channel members, but I don’t make use of this feature as I prefer that everyone has access to everything I make. The one thing that channel members do get is the ability to use our fancy SerenityOS and yak-shaving-themed emotes when commenting/chatting the channel.

Finally, YouTube also allows people to send “super chats” during live streams. These are irregular, but this month I have received $50 across 8 super chats (also part of the $315).

Tertiary source of income: Merch

People kept asking me for SerenityOS related merch like t-shirts and tea cups, so I ended up making a print-on-demand store using Teespring. Afterwards, people have told me that they felt much more comfortable supporting the project if they got something in return, “like a laptop sticker or something”. This makes perfect sense, but had not occurred to me before.

Monthly income from merch varies greatly. Last month it was $0 and so far this month it’s at $101. People are not going to buy the same t-shirt over and over again, so you have to put in some effort into making new designs/products available.

Non-income: Sponsored/native advertising

I’ve been approached to advertise a number of products, including (but not limited to) mobile games, VPN services, programming courses, zero-day exploit brokers, cloud platforms, etc.

So far I haven’t accepted any of these offers, as I made a rule for myself that I wouldn’t advertise something I’m not personally using.

The only thing that came somewhat close was when I started using the CLion IDE, the folks at JetBrains gave me three 1-year license codes to raffle out to my YouTube audience. Since I use CLion daily, both on and off YouTube, I would be happy to partner with them, but so far we’re just friends. :^)

Non-income: Venture capital

I’ve also been approached by a handful of folks from VC firms and while I have nothing against them, I’m not taking any meetings. I’m not interested in selling influence over the things I work on, and I’d much rather have many small donors who believe in me than one huge investor telling me what to do.

Accounting and taxes

All of the above is a fairly unusual way of making a living, so there isn’t a lot of guidance from Swedish tax authorities on how to report things, etc. I wanted to make sure that everything is done correctly, and that I pay all the taxes I’m supposed to, so I ended up hiring an accounting consultant.

They advised me to start a business, since it would make accounting much more straightforward, so in 2021 I formed Cerphus Software AB. The name comes from the imaginary software company I had as a child. :^)

I was trying to come up with a nice name for my business, and while there were many good options, I decided to fulfill a childhood dream and turn my then-imaginary "software company" into a real one.

I've just signed the documents forming Cerphus Software AB 👦💻🦌 https://t.co/LK2lPfSw7Z
— Andreas Kling (@awesomekling) June 28, 2021

I’m extremely glad that I hired an accountant. It cost a bit of money to get everything up and running, but we’ve settled into a comfortable routine, and they only need to spend a little bit of time per month on filing everything for me.

In fact, other than the occasional computer part, accounting services is the only real expense I have.

Summary & closing thoughts

As you can see, the numbers above put me at roughly $4200 this month. My wife and I live a modest life, and while taxes in Sweden are high, this is enough to break even where we are right now.

I know I could make a lot more money doing something else, but having the freedom to work on SerenityOS (and Jakt and Ladybird) in peace is worth infinitely more.

It’s incredibly humbling to have so many people support me financially so that I can continue my work. My massive heartfelt thanks to everyone who has supported me in the past and present! ❤️

In the future, I would love to be able to pay more people to work on the project. Especially now that we’re making a truly independent open source cross-platform web browser, I think there’s a lot of room to grow. I don’t know exactly how to do this yet, but when/if there’s enough support, I will find a way to restructure so I can hire people.

My Links

Ladybird: A new cross-platform browser project

2022-09-12T00:00:00+00:00

This post describes the Ladybird browser, based on the LibWeb and LibJS engines from SerenityOS.

Since starting the SerenityOS project in 2018, my goal has been “to build a complete desktop operating system to eventually use as my daily driver”.

What started as a little therapy project for myself has blossomed into a huge OSS community with hundreds of people working on it all over the world. We’ve gone from nothing to a capable system with its own browser stack in the last 4 years.

Throughout this incredible expansion, my own goals have remained the same. Today I’m updating them a little bit: in addition to building a new OS for myself, I’m also going to build a cross-platform web browser.

A browser is born

The Ladybird browser came to life on July 4th, when I recorded a video of myself making a simple Qt GUI for the LibWeb browser engine. Thanks to some recent work by Dex and others, we had LibWeb building on Linux in headless mode, so I decided to push ahead and build a simple GUI around it.

I originally imagined Ladybird as a debugging tool that made it easier for people to remain in Linux while working on LibWeb if they wanted to. It’s now two months later, and I find myself using Ladybird for most of my own browser development work.

At this point, we might as well tweak the scope from “browser engine for SerenityOS” to “cross-platform browser engine” and build something that many more people could potentially have use for some day. :^)

Note that LibWeb started back in 2019, then called LibHTML:

commit a67e823838943b31fb7cea68bd592093e197cf16
Author: Andreas Kling
Date:   Sat Jun 15 18:55:47 2019 +0200

    LibHTML: Start working on a simple HTML library.

LibJS began almost 9 months later, in 2020:

commit f5476be702009968468731df5e23cdeb68fdb6e0
Author: Andreas Kling
Date:   Sat Mar 7 19:42:11 2020 +0100

    LibJS: Start building a JavaScript engine for SerenityOS :^)

If you’re interested, you can see exactly how I started the LibJS engine, as the whole process was recorded for YouTube :^)

Basic architecture

Both LibWeb and LibJS are novel engines. I have a personal history with the Qt and WebKit projects, so there’s some inspiration from them throughout, but all the code is new. Not to mention, hundreds of people have worked on the codebase since I started it, all adding their own personal influences, so it’s definitely its own thing.

The browser and libraries are all written in C++. (While our own memory-safe Jakt language is in heavy development, it’s not yet ready for use in Ladybird.)

Here’s a rough breakdown of the current stack:

Ladybird: Tabbed browser GUI application
LibWeb: Web engine, multiple standards: HTML, DOM, CSS, SVG, …
LibJS: The ECMAScript language, runtime library, garbage collector
LibGfx: 2D graphics, text rendering, image formats (PNG, JPG, GIF, …)
LibRegex: Regular expression engine
LibXML: XML parser
LibWasm: WebAssembly parser and interpreter
LibUnicode: Unicode support library
LibTextCodec: Text encoding conversion library
LibMarkdown: Markdown parser
LibCore: Miscellaneous support functions (I/O, datetime, MIME data, …)
Qt: Cross-platform GUI and networking

LibWeb has a Platform layer where Ladybird injects Qt support code for event loops, timers, system font settings, etc. We currently use Qt for networking in Ladybird, as the multi-process RequestServer system is not available outside of SerenityOS yet. Likewise, Ladybird is currently single-process, while the SerenityOS browser is process-per-tab. All of this is temporary and will change over time.

License & “business model”

Ladybird and its engine are freely available under the 2-clause BSD license. You cannot buy influence over the project, but you can improve the browser by participating in development!

I’m personally working on these projects full time since 2021 thanks to my generous supporters. If you like what I’m doing, you can help me do more of it by supporting me on GitHub Sponsors, Patreon or PayPal.

I would love to have enough money to pay others to work on Ladybird some day. At the moment, I make just enough to support my own family, but if things should grow past the point where I’m comfortable, I will look into restructuring so I can hire more help.

In addition to myself, you can already directly sponsor Linus Groh and Sam Atkins who both do a lot of excellent browser work.

A note on maturity

Please note that we’re still early in development, and many web platform features are missing or broken. It’s going to take a long time before Ladybird is ready for day-to-day browsing.

We’re very much in the “make it work” part of the “make it work, make it good, make it faster” cycle. As such, we tend to focus a lot more on correctness and feature support rather than optimization. Performance work happens mostly at the architectural level, although targeted optimizations that relieve particular pain points do also happen.

Please note that this is not a product announcement or release, but more of a personal announcement that I’m adding “a truly independent cross-platform browser” to my list of personal goals. It’s also an invitation to anyone who might be interested in working on a completely new browser. :^)

As you can see above, we do pass the classic Acid3 standards test, which covers a bunch of basic CSS layout features, and various DOM/HTML APIs. However, the test does not cover many of the features used on the web today (like CSS flexbox, CSS grid, etc.)

Fidelity of modern websites in Ladybird is steadily improving, but you’ll often see lots of layout and compatibility issues. For example, here’s Reddit right now:

Community acknowledgment

Sometimes people write articles saying I’m “single-handedly” doing this or that. I’m not! Both Ladybird and SerenityOS are community efforts that hundreds of awesome people are pouring their heart and soul into. I write a lot of code, and I do my best to cheerlead for the community, but I absolutely wouldn’t be here without them!

FAQ

Q: Which platforms will Ladybird support?

So far, we’ve seen it running on Linux, macOS, Windows (WSL) and Android. The Linux version is definitely the most tested.

Since the libraries come from SerenityOS, they’re already self-contained, and we only need Qt to help us with GUI and networking. This makes the browser quite portable, and in theory we could run wherever Qt runs. In practice, we’ll see what happens.

Q: When will Ladybird be ready for use?

I don’t know. It depends on what you consider “ready”, but I’d expect a few more years of development before we have something solid. You can accelerate this process by participating in development and/or supporting our developers financially.

Q: How can I participate in development?

The most important work ahead of us is fixing bugs and adding missing features to LibWeb and LibJS. If you try opening your favorite website in Ladybird, you will find bugs! To participate in development, figure out the bug and fix it. Development discussion primarily happens in the #browser and #js channels on our Discord server, so come join us there.

Q: I found a website that does not work! Where do I report this?

At this point, there are way more websites that don’t work than websites that do. We’re not yet at the point where reporting individual site issues makes sense.

That said, if you’re going to actually work on fixing the problems, feel free to track them using GitHub issues if that helps you.

Q: Do you have a JavaScript JIT compiler?

No, we have a traditional AST interpreter that is being replaced by a bytecode VM. You can track the LibJS test262 score for both backends here. I’m not convinced that the complexity and security burdens of a JavaScript JIT are reasonable, and given recent developments like Microsoft Edge’s Super Duper Secure Mode, I’m interested in pushing for best-effort JIT-less performance while keeping the codebase simple.

Q: I opened Acid3 in my browser and I only got 97/100. What’s wrong?

You’re using an older version of the test that does not reflect settled web standards. The up-to-date version is here: http://wpt.live/acid/acid3/test.html.

Q: Why bother? You can’t make a new browser engine without billions of dollars and hundreds of staff.

Sure you can. Don’t listen to armchair defeatists who never worked on a browser.

Contact

If you’re interested in working on Ladybird, LibWeb, LibJS, or any part of the supporting stack, you can find us on the SerenityOS Discord. :^)

Memory safety for SerenityOS

2022-05-19T00:00:00+00:00

This post describes how we’re going to achieve memory safety in SerenityOS.

After visiting my nephews for easter, I spent the drive back home thinking about the future they will grow up in. What will their computers look like? What kind of software will they use? Will any of my code still be running?

When I started the SerenityOS project in 2018, I used C++ for everything, simply because it was the language I was most comfortable with. It was the right choice at the time, as it allowed me to bootstrap the project (and a community) very quickly and efficiently.

In the time since then, SerenityOS has grown larger and more complex, and we recently passed 700 individual contributors! It’s far from a one-man hobby project at this point.

When thinking about the future, I would love for SerenityOS to be around in 30 years, when my nephews are my age (and I’m an old greybeard!)

While I believe our community and system architectures are strong enough to sustain development for years to come, I no longer believe that C++ is the right language for us.

As much as I enjoy using it, the lack of memory safety in C++ means that we’ll always have bugs that could have been avoided. I’m tired of this, and I don’t want to spend the next decades of my life debugging more avoidable bugs.

To improve the longevity of SerenityOS, we need to make the system memory-safe.

So how do we achieve memory safety?

I spent a few weeks exploring the current landscape of memory-safe systems languages. I learned a handful of new ones so I could see how they work, and understand what they do to achieve safety.

I tried rewriting parts of SerenityOS in different languages, and while there were some interesting options, they all came with idiosyncratic limitations and dependencies that made them unsuitable for adoption.

Since the beginning, SerenityOS has been about making everything ourselves, for fun, for love of programming, for control, for performance, for vertical integration, etc.

In fact, the main thing we haven’t made ourselves is a programming language.

Yak-baiting a friend

Throughout this process, I’d been talking to my friend JT and sharing the struggle I had with the various languages. After talking their ear off about why some language wasn’t a good fit, I got this intriguing message:

NOTE: "yakbait" is SerenityOS slang for baiting someone into shaving your yak

JT went on to suggest that instead of settling for an existing language, we could design a new language by simply stealing the stuff we liked from other languages, and skipping the stuff we didn’t like or need.

To simplify incremental adoption, the new language would transpile to C++, which could then easily interact with our existing code.

I was hooked. Transpiling to C++!? I didn’t even realize that was an option!

We decided to name it Jakt (Swedish for “hunt”). What followed was 2 weeks of intense compiler bootstrapping with JT.

The language is now at a point where I feel comfortable telling you that we’re working on it, but it’s still a long way from “ready”.

Jakt

So, let’s take a look at Jakt! It’s…

Object-oriented
Safe by default
Paranoid about integer overflow & truncation
Immutable by default
Young and immature, not ready for anything serious

The current Jakt compiler is written in Rust and spits out C++. Development happens in the jakt repository on GitHub.

A little Jakt program

class Language {
    name: String
    age_in_days: i64
    
    function greet(this) {
        println("Hello from {}!", this.name)
        println("I am this many days old:")
        for i in 0..this.age_in_days {
            println(":^)")
        }
    }
}

function main() {
    let jakt = Language(name: "Jakt", age_in_days: 14)
    jakt.greet()
}

Memory safety

So how does Jakt achieve memory safety? Through a combination of these techniques:

Automatic reference counting of all class instances.
Runtime bounds checking of arrays and slices.
No dereferencing raw pointers in safe (default) code. unsafe keyword for situations where it’s necessary.
weak references that get emptied on pointee destruction.

Roadmap

Here’s a very fluffy 10 year roadmap for this project:

Bring up basic language features.
Improve static analysis in the compiler.
Write an auto-formatter for Jakt.
Rewrite the Jakt compiler in Jakt.
Integrate Jakt with the SerenityOS build system.
Incrementally rewrite SerenityOS in Jakt.
Stop transpiling to C++ and generate native code directly.

FAQ

Q: Why not just use an existing language?

I have nothing bad to say about other languages. This is simply the option that makes the most sense for SerenityOS, which is fundamentally about having fun and implementing everything ourselves.

Q: Why does Jakt have flaws?

It’s two weeks old.

Q: When will Jakt be done? Will it hit 1.0?

It will evolve with SerenityOS, so as SerenityOS matures no doubt Jakt will as well.

Q: Why ARC (automatic reference counting) instead of a borrow checker?

ARC allows the language to feel lightweight without constantly asking the user to make decisions about memory management.

There’s a little bit of overhead from maintaining reference counts, but we’re betting that the comfort gained will outweigh the cost.

Q: What about iterator invalidation?

We’re discussing a number of approaches, but have not settled on one yet.

Q: What about thread safety?

Jakt currently does nothing to enforce thread safety. We have not started looking at this area yet.

Contact

If you’re interested in working on Jakt, you can find us on the SerenityOS Discord. :^)

I quit my job to focus on SerenityOS full time

2021-05-28T00:00:00+00:00

Hello friends! :^) Let me tell you a story…

It was October 2018 and I had just completed a 3-month rehab program at a state addiction clinic in Sweden. I was unemployed, staying with family, and had basically nothing going on.

With no drugs or other vices to pass the time, the days seemed impossibly long. I struggled to find activities to fill them. I enrolled in school for a while, but it wasn’t for me this time either. Eventually I turned to programming, since it’s always been my big interest in life.

Until that point, my career had been focused on web browsers (WebKit at Apple & Nokia). However, I had always been interested in low-level things so I began tinkering with some of that. I wrote a little ELF executable parser.. And an Ext2 filesystem browser.. And a little GUI framework with an event loop..

Out of this tinkering, an operating system began to take shape. I chose the name SerenityOS because I wanted to always remember the Serenity Prayer. I was quite worried about my future at the time, and I figured that this name would help me stay on the good path.

My general idea was to build my own dream system for daily use. It would be a combination of my two favorite computing paradigms: the 1990s GUI and the no-nonsense command-line of late-2000s Unix.

While at Apple, I really enjoyed how most of the software was made under one roof. Not only did this enable super tight integrations, but it made the system extremely hackable for its developers, and you could always find the experts somewhere nearby. I thought I could try bringing that same feeling to the open source world, so I decided that SerenityOS wasn’t going to be a patchwork of packages – no, we’re building everything ourselves! From kernel to web browser, and everything in between.

So I started hacking, day after day. Week after week. Month after month. The project became my rock as I slowly learned to navigate life again.

I decided to record some of my programming sessions and posted them to my YouTube channel. I was very uncomfortable with this at first, but I kept at it since I liked the feeling of just being myself and letting people see me, instead of putting on a mask and pretending everything was fine.

Over time, more people discovered my little project (and my little channel), and many found something that really resonated with them. Since those humble beginnings, it has grown into a vibrant open source community with hundreds of contributors from all over the world. To say that this has been an amazing journey would be an understatement, yet in many ways we are still only getting started.

Until now, I’ve been juggling SerenityOS as a side project while also having a full time programming job.

That all changes today! I just wrapped up my last day at work, and I’m no longer employed. Instead, I will be focusing on SerenityOS full time starting right now! :^)

This is all made possible by the extremely generous support I’m receiving from folks via Patreon, GitHub Sponsors and PayPal! I feel super fortunate to have the trust & support of so many people. Thank you all so much!!

At the time of writing, I’m receiving a bit over $2000 in donations per month. There’s also a modest amount from YouTube (say $150/month), as well as sales of SerenityOS merchandise (another $100).

This isn’t yet enough to fully sustain me and my family, but it’s close enough that I decided it’s time to take a chance and see what happens!

As you probably understand, I’m not trying to get rich by doing this. I’m just a human being trying to stay sane and healthy, and it just so happens that my therapy/self-care project resonates with thousands of people, many of whom want to support it and see where it goes.

It’s truly an honor to find myself in this position, and I promise that I will continue doing my best. Thank you so much for reading this post. I’m gonna have a nap and then we’ll start fresh in the morning! :^)

Andreas Kling

Smarter C/C++ inlining with attribute((flatten))

2020-04-27T00:00:00+00:00

This post describes a compile-time technique for getting the benefits of aggressive inlining in hot code while protecting cool code from its downsides.

Hello friends!

A common technique for improving performance of hot code in C/C++ is to inline the hottest functions called. While it often helps make things faster, there are some downsides to inlining. Let’s quickly review the pros & cons:

Pros of inlining:

Removes function call overhead (yay!)
May reveal additional optimization opportunities (sometimes yay!)

Cons of inlining:

Increases program size (boo!)
May reduce cache locality (sometimes boo!)
May increase build times (boo!)

When compiling with optimizations, the compiler usually makes pretty reasonable choices about which functions to inline. It uses a combination of heuristics, with function size being the most important one AFAIK.

Manual inlining with `attribute((always_inline))`

However, sometimes you know some code is hot and the compiler has no idea. This is usually when __attribute__((always_inline)) comes in. If you add this attribute to a function, that function will now be inlined wherever it is called, even when the compiler would normally have dismissed it as too large. (Note that there are exceptions to this, and some functions cannot be inlined.)

Here’s a contrived example of a very common scenario in larger codebases:

__attribute__((always_inline)) inline void do_thing(int input)
{
    // this code is always inlined at the call site
}

void hot_code()
{
    // the program spends >80% of its runtime in this function
    while (condition) {
        ...
        do_thing(y);
        ...
    }
}

The above is all well and good, but what happens when do_thing() is a popular function that gets called a lot?

void cool_code()
{
    // the program spends <5% of its runtime in this function
    ...
    do_thing(a);
    do_thing(b);
    do_thing(c);
}

Now the cool_code() function gets three copies of do_thing() inlined into it, invoking all of the cons from the list we made above (larger program size, worse cache locality, longer build time.)

Targeted flattening instead of global inlining

Now for the trick! Both GCC and Clang support __attribute__((flatten)). Putting it on a function causes all of its callees to be inlined into it. It’s dead simple.

void do_thing(int input)
{
    // this code is not always inlined at the call site
}

__attribute__((flatten)) void hot_code()
{
    // the program spends >80% of its runtime in this function
    while (condition) {
        call_something();   // inlined!
        do_thing(y);        // inlined!
        other_thing();      // also inlined!
    }
}

void cool_code()
{
    // the program spends <5% of its runtime in this function
    ...
    do_thing(a);            // not inlined!
    do_thing(b);            // not inlined!
    do_thing(c);            // guess!
}

Note: Functions with __attribute__((noinline)) will not be inlined. The same goes for functions where the compiler can’t see the body.

In conclusion

__attribute__((flatten)) lets you opt in to the pros of aggressive inlining on a per-function basis, while protecting the rest of your program from the cons!

Until next time! :^)

Andreas Kling

Making myself uncomfortable again

Where I am today

Writing: a new frontier

MutexProtected: A C++ Pattern for Easier Concurrency

Barebones example in C

Improving it with a C++ RAII class

Adding lambdas to the mix: introducing MutexProtected

Final notes

Excellence is a habit, but so is failure

How SerenityOS declares ssize_t

Background

How we declare them

How others declare them

15 Minutes Every Day

How I make a living working on SerenityOS

Introduction

“Okay, but how do you actually make a living?”

Main source of income: Individual sponsorships/donations

Secondary source of income: YouTube

Tertiary source of income: Merch

Non-income: Sponsored/native advertising

Non-income: Venture capital

Accounting and taxes

Summary & closing thoughts

My Links

Ladybird: A new cross-platform browser project

A browser is born

Basic architecture

License & “business model”

A note on maturity

Community acknowledgment

FAQ

Q: Which platforms will Ladybird support?

Q: When will Ladybird be ready for use?

Q: How can I participate in development?

Q: I found a website that does not work! Where do I report this?

Q: Do you have a JavaScript JIT compiler?

Q: I opened Acid3 in my browser and I only got 97/100. What’s wrong?

Q: Why bother? You can’t make a new browser engine without billions of dollars and hundreds of staff.

Contact

Memory safety for SerenityOS

So how do we achieve memory safety?

Yak-baiting a friend

Jakt

A little Jakt program

Memory safety

Roadmap

FAQ

Q: Why not just use an existing language?

Q: Why does Jakt have flaws?

Q: When will Jakt be done? Will it hit 1.0?

Q: Why ARC (automatic reference counting) instead of a borrow checker?

Q: What about iterator invalidation?

Q: What about thread safety?

Contact

I quit my job to focus on SerenityOS full time

Smarter C/C++ inlining with __attribute__((flatten))

Manual inlining with __attribute__((always_inline))

Targeted flattening instead of global inlining

In conclusion

Adding lambdas to the mix: introducing `MutexProtected`

Smarter C/C++ inlining with attribute((flatten))

Manual inlining with `attribute((always_inline))`