A common challenge in software engineering for the cloud is to understand the performance of applications. Applications are often composed of many services and run on shared infrastructure. Besides the inherent performance variability of cloud services, the performance of applications can be affected by many factors, such as the the network, the application code, and user behavior.
To help developers understand the performance of their applications, and predict performance problems upfront, we have developed an idea dubbed Feedback-Driven Development (FDD)(Cito et al., 2015). The idea is to bring performance feedback into the Integrated Development Environment (IDE) of developers. This way, developers can see the performance of their applications while they are developing them, and can take corrective actions before even deploying.
Conceptual overview of feedback-driven development (FDD)
In FDD, runtime performance data is collected from, for example, application performance monitoring tools such as Dynatrace. This data is then attached to elements in the abstract syntax tree (AST) of the application. This allows us to not only visualize runtime data in the IDE, but also reason about the impact of simple code changes or additions.
We developed these ideas further in a proof-of-concept prototype called PerformanceHat(Cito et al., 2015). PerformanceHat is a plugin for the Eclipse IDE.
Screenshots of PerformanceHat in action
In a controlled experiment, we were able to demonstrate that developers using PerformanceHat were able to identify performance problems faster and more accurately than developers without the tool (Cito et al., 2019). Our ideas on feedback-driven development have been picked up by practitioners on multiple occasions. For example, Adrian Colyer has featured our work on his blog The Morning Paper, in addition to appearing in some other talks and media coverage.
Results of a controlled experiment evaluating PerformanceHat
Jürgen Cito, Philipp Leitner, Harald C. Gall, Aryan Dadashi, Anne Keller, and Andreas Roth
In Proceedings of the 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software (Onward! 2015) , Pittsburgh, Pennsylvania, United States, May 2015
@inproceedings{cito:15:2,author={Cito, Jürgen and Leitner, Philipp and Gall, Harald C. and Dadashi, Aryan and Keller, Anne and Roth, Andreas},title={Runtime Metric Meets Developer - Building Better Cloud
Applications Using Feedback},booktitle={Proceedings of the 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming \& Software (Onward! 2015)},year={2015},pages={14--27},location={Pittsburgh, Pennsylvania, United States},publisher={ACM},address={New York, NY, USA},}
@inproceedings{cito:19,author={Cito, Jürgen and Leitner, Philipp and Rinard, Martin and Gall, Harald},title={Interactive Production Performance Feedback in the IDE},booktitle={{Proceedings of the 41st International Conference on Software Engineering (ICSE)}},pages={971--981},publisher={{IEEE} / {ACM}},year={2019},doi={10.1109/ICSE.2019.00102},}
