Has anyone been curious about this simple thing? Why is not there a well known place to read or write reviews for open source software? I don’t mean those reviews as a consumer, but as the engineer who would use the open source software in their projects. In my opinion GitHub is a good choice to have reviews and ratings available, because it hosts most of the open source projects there. Due to the lack of reviews in GitHub, it’s pretty hard to discover and determine which program or library to use, there are always many different pieces of implementations for a single purpose. Even for some very niche function that there is only one repository available, I still find it difficult to make the decision without easy to find reviews.

Although no such a place to host engineering reviews of software, we are still lucky enough to read reviews which most engineers compare and write in their own blog. Sometimes it’s inconvenient to find these reviews with search engine, in another hand, reviews are decentralized in different places, and it will prevent censorship. I do appreciate those reviews, they are very important to the open source community. Programmers are the consumers of open source libraries and programs, consumers’ reviews always matters, they help to promote the software, they also act as some kind of documentation to help programmers learn the pros and cons of the library, and they are important to help the maintainers understand the issues then to improve the products. I do a lot of open source stuffs and I’m also one of those who compare and review open source programs in my blog.

Something interesting happened today because one of my reviews of WebRTC SFU servers. In a post written weeks ago, Kraken: High Performance WebRTC SFU, I have some not good reviews on Janus, mediasoup and Jitsi. And the authors of mediasoup and Janus come to the GitHub repo of kraken to yell at me “You are disrespectful. It’s just that you suck” blah blah.

To be honest, I haven’t realized my words are bad or too critical though, I just wrote about my using experiences of those software without emotion, and I think this is the right way to write reviews. However now in this post I’m not sure whether I’m emotional.

Let’s start with that piece of delicious soup, I only have three sentences on it.

1. They are designed too obscure and both recommend use of their client libraries.
2. Too much to learn about them and we suspect that they would perform better even after we mastered their usage.
3. And again, their primary languages are difficult for us to maintain the code and do urgent fixes.

I believe most people will agree that the second and third points are just my first impression on a thing, I feel it difficult, and not implemented in my familiar language, and I’m not smart enough to integrate it to our project, besides that, nothing about the soup. The first comment on the soup’s API is something exactly on it, and it’s only my personal feelings about a project, maybe just because I’m dumb, but I still can’t even find anything in the soup documentation on how to integrate it into my Go server. Then that cook blamed me on writing shit about the soup because I only speak Go, who are you man? The Big Brother?

Then Janus, more comments than that soup.

1. We used Janus in our previous WebRTC application years ago.
2. It’s our first candidate when we choose SFU for our new project.
3. We knew its API and flow.
4. It has very good performance and stability.
5. After read through its docs and issues, we find its development is so slow.
6. Without UNIFIED-BUNDLE support yet, that’s our goal which will make service more stable and development easier.
7. And we recalled our pain experience tuning Janus RPC calls, too complicated and some unexpected behaviors to hack around.
8. The most important thing is that it’s written in C/C++ and difficult for us to maintain or improve its codebase.

The first 4 is definitely praises on Janus, and actually we recommend it to people whenever they ask for some video SFU. This time when we planned to add E2EE audio conferencing to our app, Janus was still our first choice and I searched a lot about E2EE support in Janus, and learnt Unified Plan which I didn’t understand when I used Janus about 3 years ago. I find nothing valuable on the E2EE feature of Janus, so I tried to contact the Janus company for paid services.

Because E2EE is a must for us, so I tried to learn how could we do something to add E2EE to Janus like they told me. So I read the issues at first and found the issue I submitted years ago, I remembered those hacks we need to do to fix this bug and helpless facing such a C/C++ codebase. Isn’t it truth that the development is slow? For the specific UNIFIED-BUNDLE support, I already knew that Janus has a branch for it months ago when I explored its code and guess what? Today Iminiero yelled “We’ve had a branch ready and under test for weeks.” Isn’t it slow enough? Especially that cook also implies it ‘Unified-Plan is not a “super recent and cool WebRTC feature”. It’s been there for long years.’

I love open source, I use open source, I review open source and try to help if possible. Open source is not just about code, it should also deliver good API and documentation. If you don’t deliver them, programmers may find it difficult to learn, especially when you are a library author, you are the professional one in it, you should not criticize your consumers can’t implement something in minutes. I know that most of my codebase lacks docs and so I never promote them to others and I would accept criticism on this.

I would still recommend Janus to people, but never the soup, because that arrogant cook claims he is able to implement an audio only SFU in 15 minutes but the fact is he can’t even finish a working audio conferencing example by plain WebRTC browser API without any piece of his shit soup.