Update enduser-LLM.md

content/positions/enduser-LLM.md

@@ -1,43 +1,49 @@
 ---
 position: phd
-title: PhD Position "Uncertainty Management in Software Product Lines" in collaboration with Airbus 
+title: PhD Position "End-user Programming and Large Language Model" 
 duration: 36 months
-starts: 2021-09-01
-contact: Jean-Marc Jézéquel, Mathieu Acher
-email: jean-marc.jezequel@irisa.fr
+starts: 2023-10-01
+contact: Mathieu Acher
+email: mathieu.acher@irisa.fr
 ---
 
-# PhD Position "Uncertainty Management in Software Product Lines" in collaboration with Airbus 
+# PhD Position "End-user Programming and Large Language Model" 
 
-## Topic
-A software product line is a set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way. Variability management is a core concern in software product lines, making it possible to select the right set of features to answer a well defined need. 
-However when everything is not fully known, deciding wich is the optimal solution can be rather difficult. 
-Engineers typically deal with these uncertainties by first trying to reduce them as much as possible, and then most often just ignoring them (unless they work in safety critical domains).  Recognising the presence of uncertainties can contribute to reducing their influence and increasing the level of trust in a given project. Researchers have thus started to focus on identification and modelling of uncertainties, and recognised that not all uncertainty can be traced to its origin, eliminated or accounted for.  The goal of this PhD thesis is to explore how uncertainty could be more explicitly addressed and systematically managed in the modeling of software product lines, in order to guide the process of product derivation.  
+(note: it is also possible to apply for an internship!)
 
-The PhD candidate will (1) design novel variability modelling languages capable of supporting the expressiveness of uncertainties; (2) develop automated reasoning techniques (e.g., based on statistical learning) capable of communicating insights about uncertainties; (3) propose theories and foundations for engineering variability-intensive and uncertainty-aware systems; (4) empirically assess the proposed solutions in industrial contexts and possibly on open-source projets. 
+Is it possible to program non-trivial applications and customize code without knowing much about programming? Impressive showcases of Large Language Models (LLMs) like ChatGPT suggest a positive answer [1]. Some people have even claimed that programmers will be replaced or disappear [2]. So, can end-users create working and non-trivial applications, like games, just using instructions in natural language with ChatGPT?
 
-This PhD will take place in the context of the OneWay project led by **Airbus France** to redefine the future of commercial aircraft development. 
+In prior experiments [3], we considered the game Flappy bird, using Python.
+We have created 35 sessions, with prompts, code, observations, and results (videos) with ChatGPT-3.5 and ChatGPT-4.
+Our conclusion is as follows: Formulating a prompt and systematically getting a comprehensive and playable game is not yet a reality. ChatGPT can provide impressive results, but not all times. A magic prompt is missing to make ChatGPT reliable. Many interactions are rather needed to fix issues or expand the features, sometimes in non-technical terms and with instructions out of observations of the current game. But the control of the code is not that far and seems inevitable.
 
-## Bibliography 
+In this internship, we're looking for solutions to improve the situation.
+The direction we'd like to explore is to improve the usability and the integration of ChatGPT ouputs into development environment. The back and forth between the IDE and ChatGPT is time-consuming. Moreover, some (informal) instructions of ChatGPT can be automatically applied onto the code base, limiting the user effort or the technical expertise required. The hope is to have a better feedback-loop.
 
-[Ber+18] Manuel F. Bertoa, Nathalie Moreno, Gala Barquero, Loli Burgueño, Javier Troya, and Antonio Vallecillo. “Expressing Measurement Uncertainty in OCL/UML Datatypes”. In: Modelling Foundations and Applications. 2018, pp. 46–62. 
+Hence, an approach is to develop agents capable of interacting with tools and data (compilers, interpreter, error message, etc.) in an automated manner [4]. We also plan to consider a multi-modal approach where the end-user feedbacks and visual of the resulting code is leveraged. For instance, in case of Flappy bird, one can envision to execute the program, and analyze the outcome (difficulty of playing, diversity and consistency of visual artefacts, etc.) to then improve it.
 
-[EM13] Naeem Esfahani and Sam Malek. “Uncertainty in Self-Adaptive Software Systems”. In: Software Engineering for Self-Adaptive Systems II. 2013, pp. 214–238. 
+An interesting application area concerns Web applications, where technical barriers limit contributions of end-users (or even developers)...
+We envision to have an end-user "IDE" boosted by LLM and agents.  
 
-[FC19] Michalis Famelis and Marsha Chechik. “Managing design-time uncertainty”. In: Software & Systems Modeling 18.2 (2019), pp. 1249–1284 
+[1] Bubeck et al. Sparks of AGI: early experiments with GPT-4 https://arxiv.org/abs/2303.12712
+[2] The End of Programming By Matt Welsh Communications of the ACM, January 2023, Vol. 66 No. 1, Pages 34-35 10.1145/3570220
+[3] https://github.com/diverse-project/enduserprogrammingLLM/
+[4] Communicative Agents for Software Development by Chen Qian, Xin Cong, Wei Liu, Cheng Yang, Weize Chen, Yusheng Su, Yufan Dang, Jiahao Li, Juyuan Xu, Dahai Li, Zhiyuan Liu, Maosong Sun, 2023 https://arxiv.org/abs/2307.07924v3
 
-[FSC12] Michalis Famelis, Rick Salay, and Marsha Chechik. “Partial models: Towards modeling and reasoning with uncertainty”. In: 2012 34th International Conference on Software Engineering (ICSE). 2012, pp. 573–583. 
+## Objectives
 
-[PM14b] Diego Perez-Palacin and Raffaela Mirandola. “Uncertainties in the Modeling of Self- Adaptive Systems: A Taxonomy and an Example of Availability Evaluation”. In: Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering.  2014, pp. 3–14. doi: 10.1145/2568088.2568095 
-
-[BFJMPPT21] Simona Bernardi, Michalis Famelis, Jean-Marc Jézéquel, Raffaela Mirandola, Diego Perez-Palacin, Fiona Polack, Catia Trubiani. "Living with Uncertainty in Model-Based Development". In: Robert Heinrich, Francisco Durán, Carolyn L. Talcott, and Steffen Zschaler (eds.)  Composing Model-Based Analysis Tools. Springer, 2021. 
+The outcomes can well be, but are not limited to:
+ * a review of existing works about end-user programming, LLMs and agents for software development
+ * innovative approach and architecture to orchestrate LLMs and agents together with humans' feedbacks (prompts) and traditional tools (compiler, interpreter, etc.)
+ * reproducible experiments on a set of scenarios (eg Flappy bird)
+ * assessment of the proposal
 
 ## Environment
 
 The work will be realized in the DiverSE research team, joint to the CNRS (IRISA) and Inria. 
 The DiverSE team is located in Rennes, Brittany, France. DiverSE’s research is in the area of software engineering. 
 The team is actively involved in European, French and industrial projects and is composed of 9 faculty members, 20 PhD students, 4 post-docs and 3 engineers. 
-The candidate will work more specifically with [Prof. Jean-Marc Jézéquel](https://people.irisa.fr/Jean-Marc.Jezequel/) (University of Rennes 1, DiverSE team) and [Prof. Mathieu Acher](http://www.mathieuacher.com/) (University of Rennes 1, DiverSE team).
+The candidate will work more specifically with [Prof. Mathieu Acher](https://www.mathieuacher.com/) (University of Rennes, DiverSE team).
 
 ## Skills and background