Purdue collaborates with Adobe Research on configuration optimizations for complex software
Researchers in Purdue University's School of Electrical and Computer Engineering are working on ways to help users get the best performance and reliability out of increasingly complex software. Saurabh Bagchi, a professor in both ECE and Purdue's Department of Computer Science, is leading the research. Also participating in this research is Research Scientist, Dr. Paul Wood. Bagchi says software continues to have more knobs that can be tuned in order to get the best performance out of the software. It is challenging to determine the optimal values for these knobs, especially as the characteristics of the applications change.
"Increasingly, companies are running into a problem where they have a feature-rich piece of software which has many different components to it, and it has thousands of knobs each of which can be turned to a specific value between 1 and 100," says Bagchi. "There are too many possible combinations to efficiently determine the optimal values through trial and error. So what we’ve been doing is coming up with an automatic way in which to try out combinations to get some idea of what works well in the face of dynamic workloads and unpredictable contentions."
The research is being funded through the Adobe University Research Awards Program and an NIH R01 grant. A paper on the work will be presented at the Middleware 2017 conference taking place December 11-15, 2017 in Las Vegas. This paper shows how to work with large-scale genomics data in a high throughput pipeline and continuously optimize the configuration of the software packages in the pipeline in the face of changing workload patterns.
Wood said that some of their work applies to streaming applications and has near real time requirements. Further, the applications can run on a local compute cluster or on an as-needed basis, move to a public cloud like Amazon’s AWS.
"The streaming analytics itself has many knobs to it," says Wood. "How you set those knobs will determine how valuable the streaming analytics are. We will use some of the techniques that we’ve already developed to figure out the best settings for the knobs in order to get the best performance out of streaming analytics.”
Bagchi says research on streaming analytics is in the initial stages. He says Adobe will review the work after a year and determine continued funding and involvement in the project.