Visualizing Performance of Cloud Applications

2015–2019. Concluded (but see below).

A common challenge in software engineering for the cloud is to understand the performance of applications. Applications are often composed of many services and run on shared infrastructure. Besides the inherent performance variability of cloud services, the performance of applications can be affected by many factors, such as the the network, the application code, and user behavior.

To help developers understand the performance of their applications, and predict performance problems upfront, we have developed an idea dubbed Feedback-Driven Development (FDD) (Cito et al., 2015). The idea is to bring performance feedback into the Integrated Development Environment (IDE) of developers. This way, developers can see the performance of their applications while they are developing them, and can take corrective actions before even deploying.

Conceptual overview of feedback-driven development (FDD)

In FDD, runtime performance data is collected from, for example, application performance monitoring tools such as Dynatrace. This data is then attached to elements in the abstract syntax tree (AST) of the application. This allows us to not only visualize runtime data in the IDE, but also reason about the impact of simple code changes or additions.

We developed these ideas further in a proof-of-concept prototype called PerformanceHat (Cito et al., 2015). PerformanceHat is a plugin for the Eclipse IDE.

Screenshots of PerformanceHat in action

In a controlled experiment, we were able to demonstrate that developers using PerformanceHat were able to identify performance problems faster and more accurately than developers without the tool (Cito et al., 2019). Our ideas on feedback-driven development have been picked up by practitioners on multiple occasions. For example, Adrian Colyer has featured our work on his blog The Morning Paper, in addition to appearing in some other talks and media coverage.

Results of a controlled experiment evaluating PerformanceHat

The problem feedback-driven development was built for has, if anything, become more acute. You cannot see slowness in a diff, and when the diff was written by an agent, there is no author to ask what it was supposed to cost. We are returning to these ideas in our work on performance feedback for machine-written code.

Contacts:

Prof. Jürgen Cito

Prof. Philipp Leitner