Designing for Effective Use

A Framework For Effective Use

Generative AI is an emerging technology and its full impact on higher education and society is not yet realized. This technology is still evolving and will require patience and flexibility as its impact becomes increasingly more clear.  With the ever-changing landscape brought on by continuous technological development, it can be daunting for instructors to know when it is the right time to integrate new technologies like Generative AI into their classes.  While this is an academic decision resting with the individual instructor, there are several emerging best practices that can provide guidance to those considering this move.

Many frameworks and articles discussion instructor AI readiness exist, and most focus on the same 5 areas. These are:

1
Understanding of AI

Does the instructor know what Generative AI is, how it works, and the implications of its use?

2
Ability to use AI

Is the instructor capable of using generative AI to effectively complete a specific task?

3
Effective and ineffective use

Is the instructor capable of identifying situations where GenAI would be an effective tool and are they able to identify situations where GenAI would be ineffective?

4
Impacts on Discipline and Society

Is the instructor capable of describing the positive and negative impacts the use of Generative AI may have on their discipline of instruction? On society as a whole?

5
Ethical Use

Is the instructor capable of identifying ethically challenging applications of Generative AI?

If the instructor does not feel confident in any of these areas it is recommended, they develop their confidence before integrating AI into their class.

When considering what influence the Curriculum has on the decision to integrate AI it is helpful to use a framework to organize this task.  Constructive Alignment is a simple framework that can be used.  Briefly stated, Constructive Alignment outlines that there are three main components for every educational activity, these are:

1
Learning Outcome

The learning outcome is developed first, and details the enduring learning that the learner is expected to demonstrate upon completion of the educational activity

2
Assessment

The assessment is the evaluation that the outcome has been achieved. It is designed specifically to measure if and how well the learning outcome is demonstrated

3
Learning Activity

This is the activity the instructor designs to support the learner in being successful on the assessment

Each of these components influences and are influenced by the other as they come together to provide an effective educational activity.

When considering the integration of AI into a course, the learning outcomes are a good starting point.  Learning outcomes help to break a course down into a handful of statements of enduring learning. While there may not be a learning outcome that specifically states “generative AI”, generative AI may impact or contribute to the knowledge or skill of a learning outcome. This would be an appropriate place to utilize this technology.

Some learning outcomes are foundational, requiring student competency for any future learning to occur. These learning outcomes would not benefit from the inclusion of Generative AI in their assessment. Other learning outcomes are more representative of the student’s practice in industry and may benefit from the inclusion of Generative AI. Most learning outcomes fall in between these extremes.  This is where the instructors Subject Matter Expertise (SME) is so important.  Only an SME will be able to identify where AI is appropriate from within the context of the discipline.  Reviewing the learning outcomes will not likely provide enough information, but it is a good starting point.  Instructors may choose to refer to a resource like the Artificial Intelligence Assessment Scale (AIAS) to support their decision making.

Much has been written about the role of GenAI in Assessments.  A particularly useful document for this discussion is the Artificial Intelligence Assessment Scale (AIAS) written by Furze and MacVaugh (2024).  Basically, the AIAS suggests moving from a binary YES/NO approach to a more nuanced 5 stage approach.

These are:

This level ensures that students rely solely on their knowledge, understanding, and skills. This is good for developing critical foundational knowledge that students will draw on later in their programs.

Language for Students “AI must not be used at any point during the assessment.”

This means that the AI can be used in the assessment or assignment for brainstorming, creating structures, and generating ideas for improving work. The learner comes up with the ideas and how to put those ideas together, while the AI may support idea expansion or outline development.

Language for Students: “NO AI content in the final submission”

The student creates initial work, then the AI may be used to make improvement to the clarity or quality of the work to improve the final output, however, no new content can be created using the AI.

Language for Students: “AI can be used, but your original work with no AI content must be provided in an appendix”

The AI is used to complete certain elements of the task with the learner’s providing discussion or commentary on the AI generated content. This level requires critical engagement with AI generated content.

Language for Students: “You will use AI to complete specified tasks in your assessment. Any AI created content must be cited.”

The AI is seen as a collaborator in order to meet the requirements of the assignment. This collaborative approach with AI enhances creativity. A student reflection on their experience using AI in this manner may support grading for this assignment. This level reflects the expected tool use in the learner’s career.

Language for Students: “You may use AI throughout your assessment to support your own work and do not have to specify which content is AI generated.”

Furze’s model encourages reviewing planned assessments, focusing on the critical learning and being deliberate in allowing GenAI to support that learning.  Again, it is the ability to look through the broad strokes of traditional assessments, to determine which approach with GenAI can support without hindering the learning that makes the SME so critical for the integration of GenAI into the learning.  Once this determination of appropriate use has been made it is important to discuss it with students and to include it in your syllabus. We have provided sample syllabus statements later in these guidelines.

Please note if you opt for “Required AI use” you MUST use the KPU Notice of Use statement <Notice of Use – Required Generative Artificial Intelligence in this Course : TEACHING & LEARNING COMMONS KNOWLEDGE BASE>.

Traditionally, learning activities teach to a final product that will often be used as an assessment.  Looking at the learning as a whole can exacerbate the challenge of identifying where and when GenAI may be appropriate in the learning.  For example, a popular traditional learning assessment may be “write a paper on a theory presented in class”.  Taken as a whole, it is difficult to see where in this assessment that GenAI could be helpful. Pulling this activity apart, however, gives new insight. For example, “write a paper on a theory presented in class” could be expressed as the following 15 steps.

  1. Choose a theory discussed in class that interests you.
  2. Review your class notes and any related literature on the chosen theory.
  3. Outline the main concepts and arguments related to the theory.
  4. Conduct additional research to gather supporting evidence and examples.
  5. Develop a thesis statement that articulates your main argument or perspective on the theory.
  6. Create an introduction that provides context and presents your thesis.
  7. Write the body sections, ensuring each paragraph focuses on a specific aspect of the theory.
  8. Integrate supporting evidence and examples into your discussion.
  9. Analyze and critique the theory, discussing its strengths and weaknesses.
  10. Conclude by summarizing your findings and reiterating the significance of the theory.
  11. Edit and revise your paper for clarity, coherence, and grammar.
  12. Format your paper according to the required style guide (e.g., APA, MLA).
  13. Prepare a bibliography or works cited page that includes all sources used.
  14. Proofread the final draft to catch any remaining errors.
  15. Submit your paper by the deadline.

With the activity expanded in this way, it is much easier to assess each step and determine if GenAI may support the learning process. For example:

  • Step one, “Choose a theory discussed in class that interests you.”  Would not be an appropriate choice for AI.  This is a decision that rests with the learner alone.  AIAS #1 – No AI
  • Step 14, “Proofread the final draft to catch any remaining errors” may be appropriate for AI use.  AIAS #3 – AI-Assisted Editing

Presenting an assignment in this way also makes it easier for the learner to understand what it expected of them.  Not every assignment needs to break down into fourteen steps. This is why the instructor as an SME is so valuable to the process. They will be uniquely positioned to review the steps, assess if they need to be presented to the learner, determine if GenAI is appropriate and if so, at what level of the AIAS.  The following table illustrates and example of this.

Write a paper on a theory presented in class
StepTaskAIAS Level
1Choose a theory discussed in classAIAS Level 1
2Outline the main concepts and arguments related to the theory.AIAS Level 2
3Conduct additional research.AIAS Level 4
4Develop a thesis statementAIAS Level 3
5Analyze and critique the theory.AIAS Level 4
6Format your paper in APA.AIAS Level 1
7Proofread the final draftAIAS Level 3
8Submit your paper by the deadline.AIAS Level 1

Taken as a whole, this assignment might be permitted to use GenAI or not. This would be based on the whim of the instructor as there isn’t a clear rational either way.  Broken down, the assignment has a clear rational for five areas where is GenAI use is appropriate and how it may be used. Additionally, three areas have been clearly identified where GenAI is demonstrably not appropriate and why.  Breaking an assignment down like this provides an instructor with a much clearer approach to guiding or limiting the use of GenAI.  The level of permitted use should be noted in the assignment description and of course in your syllabus generative AI section. Sample wording is provided in the official “Syllabus Notice Language” and in “communication with students” to assist you.

Not every assignment will need to be broken down in this detail.  A learner with no experience in the effective use of GenAI may require this level of detail. They will need guidance to learn what is effective use, ineffective use and where this breeches academic integrity.  A student who has developed skills that allow them to use GenAI effectively may only need the assignment title and permission to use GenAI.  Assignments in 1st and 2nd year courses would benefit from being broken down with AI guidance, while courses in 3rd and 4th year should focus more on the learner integrating their AI skill into full assignments or activities that reflect what they will be accountable for in industry.  Much of this is guided by student readiness.

Similar to instructors, learners need to be ready for GenAI before they can be expected to use it effectively.  The categories are very similar:

1
Understanding of AI

Does the student know what Generative AI is, how it works, and the implications of its use?

2
Ability to use AI

Is the student capable of using generative AI to effectively complete a specific task?

3
Effective and ineffective use

Is the student capable of identifying situations where GenAI would be an effective tool and are they able to identify situations where GenAI would be ineffective?

4
Impacts on Discipline and Society

Is the student capable of describing the positive and negative impacts the use of Generative AI may have on their discipline of instruction? On society as a whole?

5
Ethical Use

Is the student capable of identifying ethically challenging applications of Generative AI?

A student who is not prepared in these areas will not be able to effectively use Generative AI. They may be able to consume the product of GenAI, but they will not be able to use it effectively, which is what we and industry want from our learners.

Another student consideration is to scaffold the use of Generative AI within the discipline alongside the scaffolded content.  Our students are going to need to understand how AI is used in the context of their discipline. This means the discipline content must be presented in learnable sections and the GenAI use must be presented in learnable sections.  In other words, Introductory discipline content is best when presented with introductory GenAI techniques, advanced discipline content with advanced GenAI content.

These approaches should help in making the decision to integrate AI into the classroom easier, more well thought out and well informed by instructor, learner and curricular need.  As importantly, it will provide a broader base of ideas and language to use in your discussions with your learners regarding the level of GenAI use in your classes.  These approaches will not alleviate the challenges of technological advancement, and for GenAI at least, these approaches should be helpful regardless of any changes the future holds.  It should also emphasize exactly why the instructor’s role in a learner’s education becomes more important with GenAI.

Resources
  • Chaaban, Youmen & Qadhi, Saba & Chen, Juebei & Du, Xiangyun. (2024). Understanding Researchers’ AI Readiness in a Higher Education Context: Q Methodology Research. Education Sciences. http://dx.doi.org/10.3390/educsci14070709.
  • Siu-Cheung Kong, Man-Yin William Cheung, Olson Tsang, (2024), Developing an artificial intelligence literacy framework: Evaluation of a literacy course for senior secondary students using a project-based learning approach, Computers and Education: Artificial Intelligence, Volume 6. 2024, https://doi.org/10.1016/j.caeai.2024.100214
  • UNESCO AI Competency Framework for Teachers. https//doi.org/10.54675/ZJTE2084
  • UNESCO AI Competency Framework for Students. https//doi.org/10.54675/JKJB9835
  • Perkins, M., Furze, L., Roe, J., MacVaugh, J.(2024). The Artificial Intelligence Assessment Scale (AIAS): A Framework for Ethical Integration ofGenerative  AI  in  Educational  Assessment. Journal of  University  Teaching  and  Learning  Practice,  21(6). https://doi.org/10.53761/q3azde36