Engineering Technician Training | Continuing Education | SIU

Prompt Engineering 

Prompt engineering is an artificial intelligence technique where the description of the task that the AI is supposed to accomplish is expressed as a series of prompts to a large language model like GPT. Because Prompts are expressed in a  natural language format, English becomes the most important programming language (to paraphrase the quote from  Andrej Karpathy of Tesla/OpenAI). Prompt engineering is simply the art of working with AI (specifically, large language models like GPT)  to get the most out of AI to solve our problems. While prompt engineering is an art, it is also a science because we need to trust the output of AI and apply the insights in a larger problem domain.  

The course Prompt Engineering for Professionals is designed for professionals and designers who want to apply prompt engineering to develop innovative artifacts that can be used professionally.

The basic idea is to iteratively improve an initial design created by AI using design thinking and systems thinking strategies in an interdisciplinary and multimodal framework. This incorporates a holistic framework of art, design, science, and engineering encapsulated in a new pedagogy (method of learning) for rapid reskilling for developers and creators through prompt engineering for building collaborative projects.

Some of the strategies we use to achieve this objective are:

Developing a process for reasoning and validation about AI output: A unique characteristic of our course is: for a given body of knowledge/professional area, our syllabus is based on developing a process of reasoning and validation using large language models (LLMs) and prompt engineering. This overcomes the limitations of GPT by correcting the 'hallucination' problem.

Emphasis on systems thinking: To consider the deployment of generative content at a professional level, we consider several systems thinking techniques, including Problem formulation; Incorporating prior knowledge; How to design and evaluate your solution; How to evaluate sources and references, including data sources; How to test multiple possibilities and pathways, and How to evaluate responses from GPT. We use critical thinking, metacognition, managing cognitive dependencies, and higher-order thinking / creative thinking skills.

Emphasis on collaboration with AI: Uniquely, the course teaches you strategies and techniques to collaborate with people and AI to create generative AI applications.

Based on a capstone project: The course is based on a capstone project developed throughout the course.

Prompt engineering is one of the hottest topics today. However, despite the opportunities, it's a complex and confusing subject for professionals. There are many questions before you can apply prompt engineering to your domain of expertise to produce valuable and innovative artifacts. In this context, we refer to an artifact as any content (text, image, or code) created in collaboration with generative AI.

Why is this so?

Unlike traditional AI, Generative AI is a 'work in progress/unfinished product.' Traditionally, users did not engage with unfinished AI products - nor were they responsible for the outcomes of unfinished AI products. For example, a regression model is accessed through an API as a finished service in machine learning. This factor impacts your choices as a developer/designer of an AI system based on generative content. By 'work in progress,' we mean that generative content is the first iteration of something a human can use in a finished product. 

Course Structure:

The course commences in October, encompassing a total of eight modules. Each module will be delivered through live three-hour online sessions. The curriculum is designed to provide a comprehensive understanding of the various aspects of prompt engineering, including:

• Foundations of AI and large language models
• Prompt engineering design and workflow
• Prompting strategies and techniques
• Challenges and complexities in prompt engineering
• Application of AI prompts across diverse industry verticals
• Ethical considerations and future developments in the field
• Design thinking and systems thinking for creating AI applications through prompt engineering

By offering an in-depth exploration of prompt engineering and its practical applications, we aim to empower participants with the knowledge and skills necessary to excel in AI-driven solutions across many domains.

After this course, you should be able to iteratively improve an initial design created by AI using design thinking and systems thinking strategies in an interdisciplinary and multimodal framework.

Modules:

• Foundations of Artificial Intelligence using Large Language Models (LLMs)

In this module, we delve into the foundations of Artificial Intelligence using Large Language Models (LLMs) such as ChatGPT. Key topics covered will include:

• Large Language Models: An introduction to LLMs and their significance in AI.
• LLM technology foundations: A deep dive into the underlying technologies that empower LLMs, including transformers.
• LLM options: A survey of various LLMs available in the market and their unique capabilities.
• Components of LLMs: An exploration of the building blocks constituting an LLM, such as attention mechanisms and tokenization.
• LLM architectures: A comprehensive overview of different LLM architectures and their strengths and weaknesses.
• The overall workflow of LLMs: A step-by-step guide to training, fine-tuning, and deploying LLMs in real-world applications.
• Multimodal approaches and GPT4 implications: An analysis of how multimodal approaches could shape the future of LLMs, particularly with advancements like GPT4.
• Prompt Engineering Tools: A showcase of tools and techniques available for practical, prompt engineering in LLMs.

We aim to provide a clear and informative guide to understanding the foundations of LLMs, equipping students with the knowledge and skills necessary to harness the power of these advanced AI models. By gaining a comprehensive understanding of LLMs, their architecture, and their applications, students will be well-prepared to design, develop, and optimize AI solutions to address a variety of challenges across various domains.

• Prompt Engineering Tools:

In this module, we explore various tools and platforms that facilitate prompt engineering and enhance AI capabilities, including:

• OpenAI toolsets: A range of cutting-edge tools such as GPT3, GPT4, and DallE
• Azure OpenAI services: AI solutions and services powered by Microsoft Azure and OpenAI collaboration
• Copilot and Copilot 365: Advanced AI-driven code completion and assistance tools
• AutoGPT: An AI tool for automated prompt engineering and optimization
• Text to Image AI: A collection of powerful tools that convert textual input into visual output and that can be effectively prompted via LLMs, featuring:
  • Midjourney
  • Playground AI
  • Stable Diffusion (Automatic1111)
  • Blockade Labs
  • RunwayML
  • Microsoft Designer

By familiarizing themselves with these prompt engineering tools, students will be well-equipped to harness the power of AI in various applications, from text generation to creative visualizations.

• The Prompting Mindset:

In this crucial module, we delve into the core strategies of prompt engineering, emphasizing the importance of fostering a mindset that enables effective communication with AI systems and accurately predicts their thought processes to yield optimal responses. Key topics covered include:

• Cultivating a mindset for prompt engineering: Techniques and best practices for developing the mental framework necessary to excel in prompt engineering.
• Engaging with AI and anticipating its thought process: Strategies for interacting with AI models and foreseeing their cognitive patterns to optimize the responses generated through skillful prompt engineering.
• Experimentation and iteration in prompt engineering: Approaches for continuously testing and refining prompts to achieve improved performance, incorporating feedback loops and ongoing adjustments to ensure the highest quality outcomes.
• Ethical considerations in prompt engineering: Guidelines and principles for ensuring that AI models generate responsible, unbiased, and fair responses, addressing potential ethical challenges and promoting transparency in the AI development process.

By mastering the prompting mindset and honing these techniques, students will be well-prepared to engage with AI systems and achieve exceptional results across a wide range of applications. In addition, students will develop a strong foundation in prompt engineering and gain a deeper understanding of the iterative nature of the process and the importance of ethical considerations when working with AI systems.

• Prompt Engineering Design and Workflow:

This module delves into the intricacies of designing and implementing AI applications from inception to completion. Key topics covered include:

• Identifying the starting point for application development: Pinpointing the initial steps and considerations for embarking on an AI project.
• Mapping out the entire workflow: Outlining the comprehensive development process and the stages of bringing an AI application to life.
• Training LLMs and the methods involved: Exploring various techniques and approaches for effectively training Large Language Models.
• The process of Prompting and techniques for Fine-Tuning: Examining the art of crafting prompts and refining AI models to achieve desired outcomes.
• Design thinking and systems thinking for generative AI.

We aim to provide a clear and informative guide, equipping students with the knowledge and skills necessary for practical application design and implementation across various AI projects.

• Prompting Strategies:

In this module, we delve deeper into the nuances of prompt engineering, building upon the foundations established in previous modules. We explore various prompting strategies and their applications, including:

• Role Prompting: Techniques for guiding AI responses by assuming specific roles or perspectives.
• Persona-based Prompting: Strategies for generating responses tailored to distinct personas or character profiles.
• Few-shot Prompting: Approaches for leveraging limited examples to guide AI model behavior effectively.
• Combining Techniques: Methods for integrating multiple prompting strategies to achieve more refined and targeted results.
• Chain of Thought Prompting: Tactics for creating a series of interconnected prompts to maintain context and continuity.
• Zero-shot Prompting: Techniques for eliciting desired AI responses without providing explicit examples or guidance.

We aim to equip students with a comprehensive understanding of various prompting strategies, enabling students to effectively harness the power of AI models in generating creative and contextually relevant responses across diverse applications.

• LLM Applications by Verticals:

In this module, we delve into the vast landscape of AI prompts specifically tailored to address various industry verticals' unique needs and challenges. We provide clear and informative examples for each sector, encompassing:

• Legal AI Prompts
• Startup AI Prompts
• Sales AI Prompts
• Content Writing AI Prompts
• E-Commerce AI Prompts
• Education AI Prompts
• Customer Service AI Prompts
• Human Resources AI Prompts
• Product Management AI Prompts
• Development AI Prompts
• Design AI Prompts
• Marketing AI Prompts
• Finance AI Prompts
• Graphic Design Prompts
• Website Design Prompts
• Artistic Design Prompts
• Game Asset Design Prompts

We aim to empower students with extensive AI prompt examples tailored to each industry's requirements and challenges. This comprehensive understanding will enable students to implement AI solutions across various domains effectively.

• Challenges in Prompt Engineering and the Future of Prompt Engineering:

In this module, we delve into the various challenges and complexities that may arise in prompt engineering. We provide a comprehensive understanding of these challenges, including:

• Handling ambiguous or unclear prompts
• Mitigating biases in AI-generated responses
• Ensuring response accuracy and relevancy
• Addressing potential ethical concerns
• Dealing with prompt overfitting or underfitting

By exploring these challenges, we aim to equip students with the knowledge and strategies to effectively navigate potential obstacles and ensure robust AI prompt performance.

Finally, we delve into the fascinating possibilities that lie ahead in the rapidly evolving field of prompt engineering. We explore various emerging trends and future developments, such as:

• Advances in AI models and architectures (AutoGPT)
• Multimodal AI systems and their potential impact
• The role of ethics and regulation in prompt engineering
• Integration of AI prompts in new application areas
• Continuous improvement in prompt engineering techniques

We aim to provide students with a glimpse of the future, inspiring students to stay ahead of the curve and adapt to the constantly changing landscape of prompt engineering.

This comprehensive course in prompt engineering with large language models is designed to empower participants with the essential knowledge and practical skills required to excel across various AI applications. Spanning diverse fields such as business, humanities, and social sciences, our curriculum aims to provide a solid foundation in prompt engineering techniques, LLM architectures, and AI-driven solutions.

By combining theoretical knowledge with hands-on experience, we aspire to cultivate a deep understanding of the intricacies and nuances of AI systems and their potential impact on various industries. In addition, our goal is to inspire and equip participants with the capabilities to harness the transformative power of AI, enabling them to drive innovation, enhance productivity, and address complex challenges facing the contemporary world.

Who is Eligible?

• XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXEligible to work in the US
• Be 21 years of age or older
• Have a high school diploma or GED (The General Educational Development Test) equivalent
• Preference will be given to residents of underserved/disadvantaged counties in Southern Illinois
• Valid Illinois driver’s license
• Illinois resident not currently employed in the infrastructure industry
• Have not completed similar training before.