Frank Schmid

Interview with Frank Schmid

It was in 1982 with a Commodore 64 (you remember that one?) when I discovered my passion for programming and computer technology—at that time, with Basic and machine language. I was 12 years old. From then on, my journey and passion continued steadily.

All along this path, the excitement and comprehension for different programming languages and architectures grew—and together with that, the eagerness to find out how to solve specific problems with the appropriate tools for the problems (languages and architectural patterns).

At the age of 16 I sold my first application. It was a complete solution for order management. All throughout school and my time at the University I dug further into software development and software architecture and designed and implemented more applications, especially for solving problems in the construction sector. I think it was those kinds of problems, and that practice, which effectively strengthened my capability to solve problems efficiently by finding the right algorithms and the best architecture. This was the ultimate preparation for my development as a software engineer.

I ran my own company for several years, in the network design, administration and software development space, before finally switching back to pure software development. I joined a much bigger company and stayed there for 10 years.

During my tenure there, I ran through various positions: senior software engineer, solutions architect, team leader, and finally, technical director. All the practical experience I already had (in languages like C++/C/Assembler/SQL/PHP/Perl, etc.), and knowing how to solve problems with the proper language and architecture from prior experience, helped me to improve in areas of software engineering that were not that important and visible before. It even turned out that cultural differences impose a problem that has to be solved—in my time as team leader, part of my team was located in China—and was an essential part of software engineering. As Technical Director, a strategic component complemented the tactical and operational ones.

After that I switched back to a small startup—Stashimi. That was nearly 3 years ago. Of course this is completely different, it is agile, hands-on. It is fun, though it actually means more work. :-) It feels like going back in time before the big company, but with a much more complete understanding of the software world.

You asked How did you get into programming?

Looking back over the years, I think "programming" is only a very small part of what crosses the gap between success and failure. It is a lot more than that—it is more a well-balanced mix of knowledge, problem solution and detection skills, target-orientation, etc. And even as you evolve further over time, you see that more and more areas come in that you initially thought not to be related with "programming" at all. For example, the personal fit of team members, which is as essential for a team's and a project's success as problem solving skills or knowledge. Also things like the best working methodology (agile/SCRUM vs. traditional) for the team.

At Nero, we had a backend that was completely built up traditionally, with servers, databases, etc. as we knew it years ago. It was completely run on-premise, i.e. on servers that we either rented or owned. Before even thinking of serverless architectures, cloud-based services like AWS, Google or Azure appeared that offered server and database hosting in the cloud.

But for a big company (at least ours), even the thought to go "hosted" had initiated discussions about data security by people who were scared about giving up any ownership of the data or just the risk of losing data. With some headwind, though, it was finally possible to get some (not really mission critical) services into the cloud.

Although the cloud based services ran without issues, enabled a better coordination of multinational teams, and were ultimately cheaper, it was not possible to really invent a completely new approach upon the cloud-based services there.

That completely changed with Stashimi. We had the opportunity to design a new backend from scratch but the point is: we were not scared by the cloud and envisioned the evolution of backends going towards technologies that are completely cloud-hosted.

With that mindset, it was only a small step to go further—a cloud-hosted serverless architecture.

We evaluated the available providers, and came to the conclusion that we'd go with AWS (I already had experiences with AWS, Microsoft,and partially Google before with the former company). They just offered everything we needed as easy-to-use and secure services. For us, that was lambda functions (with Node.js support), a cloud API, no-SQL databases (DynamoDB and ElasticSearch), SQL databases and good scaling and regional distribution capabilities, and finally the possibility to have mobile clients easily interact with the backend by using a client SDK compiled for the specific API and AWS Cognito.

We designed our complete backend based upon the capabilities that AWS offered at that time, and we continuously use new AWS serverless services (e.g. Lex) as we add features to our platform.

I started using the Serverless Framework quite early. Although the serverless services themselves worked perfectly in AWS cloud, it was a big problem that they did not have an easy way to deploy them via a CI/CD based system (at least not without creating and managing lots of configurations manually).

AWS SAM or other helpers weren't on the horizon at that time, and even now they do not offer the needed flexibility to run a real-world deployment infrastructure.

Austin's JAWS looked very promising, as well as the initial Framework versions (0.1 and 0.2). We experimented with them and set up a deployment system right away. Even though being an early adopter introduced some hurdles in regards to fluctuating features and capabilities, it made our backend deployments reliable and reproducible.

When the Framework 1.0 version appeared (which was a needed rewrite), it was a vast improvement in regards to reliability and stability. Especially the change from using the AWS REST API to deploy the service to using CloudFormation. It was definitely the right choice.

We adopted it and upgraded our projects short-term after a short testing phase.

It is the architecture at Stashimi, which is completely serverless. We have a backend that serves mobile clients and bots, and with this combination we can really see how powerful and secure a serverless infrastructure is, and how well it scales. The performance and reliability is astonishing.

As I pointed out already above in my answer to my serverless journey, one big issue that prevents especially big (and often inflexible) companies from going the right way, is the fear of the cloud in general and the missing trust of "legacy" decision makers in the cloud providers. Of course, any cloud-based solution requires the setup of a proper security strategy, but caution is quite different of fear. Even the current Facebook discussion and incidents intensify that for people who are technically not very deeply anchored in the topic.

If these problems are removed (by better information and training), there should be no obstacle to move any existing server based systems to serverless systems. Cloud providers offer lots of services nowadays that let you model _any_ architecture as serverless architecture. Deployment-wise, the Serverless Framework offers all the flexibility to provide the complete operational part.

It is like I said at the very beginning: If you know which tool (specific cloud provider service, like databases, queues—SNS or SQS, etc.) to use to solve your actual problem (the backend requirements) you can do it, even if it is not obvious at first sight.

We are full-in on AWS as I mentioned above, so we have modeled the complete backend in there and do not use other providers. This is partially historical, because AWS were the first to quickly offer new needed services, which allowed us to realize new features in a short time frame. Other providers (IMO especially Microsoft) are generally paced slower and offer the features a bit later.

There is one thing that could be improved on the AWS side: We do image manipulation in a Lambda function, and the only library that is available (in Node) in a portable way is Imagick. I also did an implementation using OpenCV (which offers a lot more functionality), but the problem here is, that native libraries have to be compiled on Amazon Linux (the same OS as running on Lambda) so that they can be invoked in a Lambda.

I've seen this problem arising already multiple times in my `serverless-webpack` project, where people also tried to upload Node modules that contain native bindings. This is even more complicated if you deploy from a non-Linux OS like Darwin or Windows.

I'd like to see a better solution in that area, maybe some AWS binary library repo that also includes bindings for the supported Lambda runtimes. Then, any Lambda could just install the packages from their repository (an AWS npm/yarn repository?) instead of the original one and it would work, independently of the runtime or deployment OS used.

Speaking of AWS as an example here, mainly because that's what we use.

Over the past few years, all services have matured and are nearly error-free now. Of course, newly-invented ones still need some time to mature, but AWS does a good job there and you can give feedback when testing new features that are still in beta.

The same is true for observability. It is possible to create a completely transparent monitoring of everything you run in their environment. You can create custom metrics easily that track the internal functionality of your implementation and display them in real-time dashboards. That way, we're able to see and react to any inconsistencies in the system quickly.

Tooling is the only point that lacks a bit behind there. It is overly complex to do things like easy deployment of bigger, dependent projects. But I do not see that as disadvantage, because there are great tools that fill this gap (e.g. the Serverless Framework). I even think it is better if the providers concentrate on the services and less on the tooling and let the tools be done and improved by people that actually build up their architectures on top of the services.

Developer experience in the serverless area is still quite sparse if you take all developers who are engaged in cloud or network based technologies. But I'm sure that with the continuing trend towards severless architectures and their advantages, this will change over time. However, it is important that there are a few strong evangelists who support and strengthen this trend until in the end, server-based systems are the exception.

There are some major key points that make a long-term transition inevitable.

There is the delegation of fixed administration and running costs to the provider—you pay as you go. The costs scale with the actual load of the system. Additionally the provider costs tend to decrease over time (at least from what I see).

Then, serverless based systems scale better and can cover load spikes more easily. Architecture-wise, serverless architectures are in general modeled more cleanly. They stay extensible and separated and are maintainable in a much better way. New services that are hard to be managed and maintained in a meaningful way appear frequently and offer a better experience (e.g. AWS Lex, AWS Comprehend, etc.)

Of course, adopting serverless architectures is not an ad-hoc thing. It requires a change in people's minds, which can be a very slow process; you might even have to wait for a new generation of software engineers to occupy senior-level positions. It is also most likely no binary decision where you just state "now we will be serverless"—same as with someone who tries to introduce agile methodologies ad-hoc and for everything, even making coffee. ;-) That would obviously not work, and only be food for objectors who are against changes at any cost.

I'm sure that we'll continue to see a large shift towards serverless systems over time. Even present-day objectors will most likely follow as the serverless architecture in the end (IMO) leads to a market advantage for the implementers.

First of all—thanks! Honestly, answering huge amounts of questions in `serverless` and my projects `serverless-webpack` and `serverless-aws-alias`, I think that there is no specific question that appears more often than others. All questions are important and should be answered properly, and most of them are on how to solve very specific problems.

In general, you can group the questions in a few categories: questions about things that are broken or missing, general architectural questions (how do I do X?) or interesting discussions about new possible features or different ways to do things. All of these categories have a different effect on the topic that is questioned, but all of them increase community knowledge. They help to drive the Serverless Framework, as well as serverless architectures, into the future.

I'm not sure I missed anything—oh, yes: The most important one:

I'm really impressed with the Serverless Framework and the community. The project has managed to attract and gather very competent and nice people from all over the world. I'd like to thank the community to make this work and happen.

The Serverless Framework helps a lot of companies and people. Having such a living community is of more value than any similar tool any cloud provider could provide.