Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Implement changes made necessary by the fix for MDL-83541 #249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mjleblanc opened this issue Mar 26, 2025 · 16 comments
Open

Implement changes made necessary by the fix for MDL-83541 #249

mjleblanc opened this issue Mar 26, 2025 · 16 comments

Comments

@mjleblanc
Copy link

After fix for MDL-83541, code runner questions are always duplicated when duplicating a quiz in a course or doing a backup and restore in same course.

@raveiga
Copy link

raveiga commented Mar 31, 2025

Yes, I have the same problem with latest version Moodle 4.5.3+

When a teacher duplicates a quiz with questions related with Coderunner it duplicates all the questions in database.

@tlock
Copy link

tlock commented Apr 3, 2025

Also, the unit tests are failing:

mod_quiz\backup\repeated_restore_test::test_restore_quiz_with_duplicate_questions with data set "coderunner" ('coderunner', 'sqr')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'3'
+'4'

mod/quiz/tests/backup/repeated_restore_test.php:423

mod_quiz\backup\repeated_restore_test::test_restore_quiz_with_edited_questions with data set "coderunner" ('coderunner', 'sqr')
Failed asserting that 1 matches expected '3'.

mod/quiz/tests/backup/repeated_restore_test.php:490

@trampgeek
Copy link
Owner

trampgeek commented Apr 3, 2025

Yeah, well. Sorry guys, but I've been somewhat blindsided by this sudden change to the question API in the middle of a teaching semester when I have 1000 students in my course. My CodeRunner development time is over the summer break (Southern Hemisphere time), not in the middle of a course.

I have pushed to the development branch code that results in CodeRunner questions now passing Moodle's repeated_restore_test but questions are still being duplicated when you duplicate a quiz. I clearly need to dig deeper but time is very limited right now so I'm afraid you'll have to wait a bit. Unless someone else chooses to fix it.

trampgeek added a commit that referenced this issue Apr 3, 2025
…the fix for MDL-83541. This commit adds the functionality required to pass the repeated_restore test, but CodeRunner questions are still being duplicated when a quiz is duplicated or a course restored. Needs a deeper dive, but time is limited.
@tlock
Copy link

tlock commented Apr 3, 2025

πŸŽ‰

@tlock
Copy link

tlock commented Apr 3, 2025

Some feedback, this unit test fails for us on v5.2.1.

Moodle: 4.5.3+ (Build: 20250328)
Php: 8.3.16, pgsql: 15.8 (Postgres.app), OS: Darwin 24.3.0 arm64
PHPUnit 9.6.18 by Sebastian Bergmann and contributors.

  1. qtype_coderunner\restore_test::test_restore
    Undefined array key "testtype"

/usr/local/git/moodle2/moodle5/question/type/coderunner/backup/moodle2/restore_qtype_coderunner_plugin.class.php:115

(
[id] => 27
[testcode] =>
[expected] => Good
[useasexample] => 0
[display] => SHOW
[hiderestiffail] => 0
[mark] => 1.000
[stdin] =>
[extra] => nfa
epsilon
)

  1. qtype_coderunner\restore_test::test_restore_from_v3_0_0
    Undefined array key "testtype"

/usr/local/git/moodle2/moodle5/question/type/coderunner/backup/moodle2/restore_qtype_coderunner_plugin.class.php:115

Array
(
[id] => 12
[testcode] =>
[expected] => Good
[useasexample] => 0
[display] => SHOW
[hiderestiffail] => 0
[mark] => 1.000
[stdin] =>
[extra] => nfa
epsilon
)

testtype not defined in: loadtesting_pseudocourse_backup.mbz

Fixed:

-                $testcase->$field = $record[$field];
+                $testcase->$field = $record[$field] ?? '';

@CyrilWendl
Copy link

It seems like the duplication issue is still not resolved, so I propose to keep this issue open if possible. Thank you!

@trampgeek trampgeek reopened this Apr 7, 2025
@trampgeek
Copy link
Owner

Certainly it should be kept open. I didn't intend to close it, sorry.

@ngandrass
Copy link
Contributor

I tried to find the exact cause of the duplication issue but haven't gotten my head around it yet. I used the latest development branch.

What I did find:

  • Only prototypes seem to be duplicated
  • The final question (indicated by prototypetype = 0) is not being duplicated
  • Having a quiz containing a single coderunner question: When duplicating this quiz multiple times, all previously duplicated prototypes are again duplicated. This results in: 1, 2, 4, 8, 16, ... prototypes after each quiz duplication.
  • Some question options are conditionally casted to other types or set to null under certain conditions. This will likely impact the question hash generation during duplicate detection.
  • tests/helper.php does not seem to contain any used prototyped question. Therefore mod/quiz/tests/backup/repeated_restore_test.php most likely does not run into this issue.
  • Excluding the noninherited_fields from the question data object generated by remove_excluded_question_data() does not fix the issue
  • A try to replicate the conditional type casts from get_question_options() and its decedents failed due to missing knowledge of the underlying data structures on my side.

Sadly I have to move on to other topics right now but I wanted to at least report on the stuff I tried and found out. Hope this helps anyone :)

@CyrilWendl
Copy link

I tried to find the exact cause of the duplication issue but haven't gotten my head around it yet. I used the latest development branch.

What I did find:

  • Only prototypes seem to be duplicated
  • The final question (indicated by prototypetype = 0) is not being duplicated
  • Having a quiz containing a single coderunner question: When duplicating this quiz multiple times, all previously duplicated prototypes are again duplicated. This results in: 1, 2, 4, 8, 16, ... prototypes after each quiz duplication.
  • Some question options are conditionally casted to other types or set to null under certain conditions. This will likely impact the question hash generation during duplicate detection.
  • tests/helper.php does not seem to contain any used prototyped question. Therefore mod/quiz/tests/backup/repeated_restore_test.php most likely does not run into this issue.
  • Excluding the noninherited_fields from the question data object generated by remove_excluded_question_data() does not fix the issue
  • A try to replicate the conditional type casts from get_question_options() and its decedents failed due to missing knowledge of the underlying data structures on my side.

Sadly I have to move on to other topics right now but I wanted to at least report on the stuff I tried and found out. Hope this helps anyone :)

Thank you so much. May I ask, for the Moodle user community not familiar with the internals, what a prototype is? And is there a way to somehow fix this issue by marking a question as non-prototype within the Moodle UI? Do you know of any other ways how users could circumvent the issue?

Thank you and kind regards

@timhunt
Copy link
Collaborator

timhunt commented Apr 9, 2025

I just want to make sure we have a link here to the key documentation:

@trampgeek
Copy link
Owner

trampgeek commented Apr 9, 2025

Thanks for looking into this @ngandrass . I had a quick look myself yesterday.

As you've spotted, the problem lies with CodeRunner's use of prototype questions. In CodeRunner a runnable question comes in two parts: a prototype that defines the type of CodeRunner question (the programming language, how it behaves and is graded etc) and the actual question. Prior to being run, the actual question is merged with the prototype. Loosely you can think of the prototype as a class definition and the question as an instance of the class.

The algorithm being used in this "fix", is to compare a raw question imported from the backup with an existing loaded-and-ready-to-run version. But in CodeRunner these are two different things, because the latter has had all the prototype attributes merged into it. This merging takes place in the following line in my get_question_options() function

$this->set_inherited_fields($options, $question->prototype);

If I comment out that line, the duplication problem goes away. (Well, it goes away in my current simple test, anyway). If I had some way of knowing that the call was coming from the module-duplicate-and-backup-restore processor I could simply skip that line. But I can't see a way to do that, except by inspecting the runtime stack. Which is just too evil, even for me.

Today I'll see if I can remove all those prototype attributes again in the remove_excluded_question_data method. It's not going to be pretty, but it's the best idea I have at present.

[*** EDIT *** Big discovery. During the call to get_guestion_options() the question has been tagged with the attribute 'export_process'. Testing for this attribute and, if found, disabling the setting of inherited fields seems to largely solve the problem. Just some edge cases to deal with :-) ]

@trampgeek
Copy link
Owner

As per the ** EDIT ** above, considerable progress has been made. I've pushed an update to the development branch which so far seems to solve the problem. I just duplicated a quiz in a course with ~2000 questions and don't seem to have any duplicates. But much more thorough testing is required.

@ngandrass
Copy link
Contributor

Thanks for your work!

I did some testing and 59cda8c seems to introduce side effects that make mod/quiz/tests/backup/repeated_restore_test.php fail. This problem is addressed by #253


After fixing this I performed a test like I did yesterday and your patch indeed seems to solve the duplication issue πŸ‘

However, since this code potentially affects question exports I did the following:

  1. Export a coderunner question from one of our production systems running the current stable release
  2. Import the question on my local dev system with the development version
  3. Make sure everything works as expected and the correct prototype is found
  4. Export the question again to Moodle XML
  5. Compare the generated XML files (master vs development)

And good news: Both XML files look identical. Therefore I'd consider this a fix πŸŽ‰


I'll throw the patched version on one of our staging systems later today and will test duplication in a big course with a large set of questions. Will report back here afterwards :)

@ngandrass
Copy link
Contributor

A colleague and myself just tested this fix on one of our staging systems, duplicating large quizzes in large courses and everything seems to work fine. Neither questions nor their prototypes get duplicated πŸ‘

@CyrilWendl
Copy link

Excellent, thank you all for your work on this. For us, the end users: can we now just install the latest CodeRunner version? And does this also fix the duplication issue not only for CodeRunner questions but also for STACK questions, which show the same unwanted duplication behaviour?

@trampgeek
Copy link
Owner

@ngandrass - thanks for the help with testing. I've merged your tweak to get the restore-test working again (thanks for that), updated the version number and fixed the failing test case (which was pretty irrelevant - attempts to import 8-year-old course backups weren't working). I've not found any further errors, and have pushed to the development branch. I'd prefer not to merge to master until I've run this code on our production server for at least a week or two.

@CyrilWendl - you will need to pull the development branch from github (not the master branch) to get these changes. And certainly they will not solve the problem with STACK questions. Every question type needs to be independently updated to implement the new API calls that they added to fix MDL-83541. I feel the process of rolling out the fix could have been better handled. I was only made aware of the API changes three weeks ago, in the middle of term time. It's perhaps fortunate that I'm currently on leave and have had time to look into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants