feat: add a command to reset user's Jobma interview state

[arslanashraf7]

What are the relevant tickets?

#1362

What's this PR do?

• Adds a command to reset user's Jobma interview state

How should this be manually tested?

• Setup bootcamp(s) and runs along with user(s) - Can be done with seed command
• Enroll user in the bootcamp
• Fill the user profile in the bootcamps and make them reach a state where they have failed the Video interview in Jobma. If you can't setup the things locally to complete the user's steps in the bootcamp, you can use an existing command to change the user's state using an existing command e.g. ./manage.py set_application_state --app-id=5 --state=AWAITING_PAYMENT
• Check that the submissions exist for the user and there are VideoInterviewSubmission and Interview records - Check in the Django Admin

Now check the following:

• InterviewAudit entries are now available in Dango Admin
• On the frontend, Bootcamp enrollment page, the user should be showing Awaiting payment state
• Now run, ./manage.py reset_application_interview_state --user=<username or email or id> --run=2 for the user and it should now delete the Interview and its related submission records, and the user on the frontend should be back to the Video Interview stage.
• Also, check the fail cases for the command. e.g. if the user's application is not in applications/constants/REVIEWABLE_APP_STATES. The command should show an error
• Also test this flow as per mentioned in the Original Ticket. Because I've refactored the logic around that a bit too.

Also, note, Application should only reset if its state is in REVIEWABLE_APP_STATES, Otherwise the reset_application_interview_state should print an error message.

Where should the reviewer start?

Set up user, bootcamp enrollment, and a state where the user has passed the Video interview step.

[pdpinch]

Now run, ./manage.py reset_application_state --user= --run=2 for the user and it should now delete the Interview and its related submission records and the user on the frontend should be back to the Video Interview stage.

It looks like the management command is actually named reset_application_interview_state which is a clearer, more specific name than reset_application_state

[cachob]

On the frontend, Bootcamp enrollment page, the user should be showing Awaiting payment state

Will the command only work if the learner is in the Awaiting payment state?

[mbertrand]

Didn't work for me, here are the commands I ran:

Creating bootcamp-ecommerce_web_run ... done
Application: 
user='mrbertrand@gmail.com', run='Seed Single Variable Calculus Run 2', state=AWAITING_PAYMENT
State change: AWAITING_USER_SUBMISSIONS -> AWAITING_PAYMENT

Application now shown as "In progress", with "Video Interview" completed, "Application Review" approved, "Payment" as final step.

Then ran:

mattbert@Matts-MacBook-Pro bootcamp-ecommerce % docker-compose run web python manage.py reset_application_interview_state  --user 2 --run 2
Creating bootcamp-ecommerce_web_run ... done
Jobma interview state has been reset for User=matt-b Run=Seed Single Variable Calculus, Sep 9 - Dec 8, 2023.

In django admin, the application state is AWAITING_USER_SUBMISSIONS . But nothing changed on the front end, video submission still exists in django admin. Tried again:

mattbert@Matts-MacBook-Pro bootcamp-ecommerce % docker-compose run web python manage.py reset_application_interview_state  --user 2 --run 2
Creating bootcamp-ecommerce_web_run ... done
CommandError: User's application is already in approved state. User=matt-b, Run=Seed Single Variable Calculus, Sep 9 - Dec 8, 2023, State=AWAITING_USER_SUBMISSIONS.

Repeated the above with a slight variation, same basic result:

mattbert@Matts-MacBook-Pro bootcamp-ecommerce % docker-compose run web python manage.py set_application_state --app-id=3 --state=AWAITING_SUBMISSION_REVIEW
Creating bootcamp-ecommerce_web_run ... done
Application: 
user='mrbertrand@gmail.com', run='Seed Multivariable Calculus Run 2', state=AWAITING_SUBMISSION_REVIEW

Video interview completed, awaiting application review

mattbert@Matts-MacBook-Pro bootcamp-ecommerce % docker-compose run web python manage.py reset_application_interview_state  --user 2 --run 4                
Creating bootcamp-ecommerce_web_run ... done
Jobma interview state has been reset for User=matt-b Run=Seed Multivariable Calculus, Sep 9 - Dec 8, 2023.

Status has been reset to AWAITING_USER_SUBMISSIONS but video submission still exists, no change on front end: "Video Interview" shown as completed, review pending.

mattbert@Matts-MacBook-Pro bootcamp-ecommerce % docker-compose run web python manage.py reset_application_interview_state  --user 2 --run 4
Creating bootcamp-ecommerce_web_run ... done
CommandError: User's application is already in approved state. User=matt-b, Run=Seed Multivariable Calculus, Sep 9 - Dec 8, 2023, State=AWAITING_USER_SUBMISSIONS.
ERROR: 1

[mbertrand]

PS I started with a clean database, ran the following first:

manage.py seed_data
manage.py configure_cms
manage.py createsuperuser

Then registered a user and applied.

[arslanashraf7]

It looks like the management command is actually named reset_application_interview_state which is a clearer, more specific name than reset_application_state

@pdpinch Thanks, I fixed that in the PR description.

Will the command only work if the learner is in the Awaiting payment state?

@cachob The command should work for all the pre-decided/declared possible reviewable app states. These are mentioned in https://github.com/mitodl/bootcamp-ecommerce/blob/master/applications/constants.py#L42-L45. Currently, they are

REVIEWABLE_APP_STATES = [
    AppStates.REJECTED.value,
    AppStates.AWAITING_PAYMENT.value,
    AppStates.AWAITING_SUBMISSION_REVIEW.value,
]

@mbertrand Thanks for trying all those possible combinations, While I was testing it locally, It worked for me and even when I tested it right now it works and I think I have the answer to why they might not be working for you.
In the command, I'm calling refresh_pending_interview_links at the end. And the actual implementation of the refresh_penfing_interview_links has some cutoff time filters which would have been returning empty queryset for your user. I'm guessing the application runs that you are running this command on probably has one or more of the following conditions:

1. Application state is in REVIEW_COMPLETED_APP_STATES or, the run start date is in the past past
2. Video interview submission step is not in pending state or its creation date is between the last 30 days.

I tested it on a data set that was created locally some time ago.

Now, I guess, In the case of the command, We should just bypass the creation duration of the steps and the statuses. What do you think @mbertrand ?

[mbertrand]

@arslanashraf7 thanks, it was the fault of cutoff date for jobma interviews:

        .filter(
            Q(submission_status=SUBMISSION_STATUS_PENDING)
            & Q(videointerviews__created_on__lte=settings.JOBMA_LINK_EXPIRATION_DAYS)
        )

Maybe the function could be modified to pass in an optional cutoff_days kwarg:

@app.task
def refresh_pending_interview_links(cutoff_days=settings.JOBMA_LINK_EXPIRATION_DAYS):
    """Recreate old pending interviews"""
    now = now_in_utc()
    cutoff_date = now - timedelta(days=cutoff_days)
    ...
    ...
    .filter(
        Q(submission_status=SUBMISSION_STATUS_PENDING)
        & Q(videointerviews__created_on__lte=cutoff_date)
    )

and then the mgmt command could call it like this:

refresh_pending_interview_links(cutoff_days=0)

[arslanashraf7]

@mbertrand Good suggestion but here are a few more things to that, I gave a quick try to your suggestion as well but I think we might have to go with a bypass approach.

Why?

• With the user in the Jobma interview failed state, Q(bootcamp_application__bootcamp_run__start_date__lte=now) would also exclude their application from refreshing
• Also, Q(submission_status=SUBMISSION_STATUS_PENDING) would exclude the user's application steps because at the point after video interview failure, the submission_status is going to be submitted instead of pending.

Based on the above, I think only tweaking the cutoff time would not work completely.

For the filters bypass approach, I'm making the refresh functionality independent of the filters and dependent upon the provided list of submissions itself so that we can just pass a list of submissions from anywhere and their video interview state could be refreshed.
[WIP] This work is being done in 21e6bdc let me know what you think.

[arslanashraf7]

@mbertrand this is ready for review again.

[mbertrand]

I am getting this error when running the mgmt command, which results in the transaction being aborted and no changes made. I have set all my JOBMA .env variables to the same as RC, are those correct? What values are you using? I vaguely recall that jobma might not be working properly on RC either. #1296

Traceback (most recent call last):
  File "/src/manage.py", line 14, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/src/applications/management/commands/reset_application_interview_state.py", line 58, in handle
    refresh_jobma_interview_submissions(user_run_application.submissions.all())
  File "/src/applications/api.py", line 131, in refresh_jobma_interview_submissions
    populate_interviews_in_jobma(submission.bootcamp_application)
  File "/src/applications/api.py", line 157, in populate_interviews_in_jobma
    create_interview_in_jobma(interview)
  File "/src/jobma/api.py", line 59, in create_interview_in_jobma
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://testing.jobma.com/interviews

[arslanashraf7]

@mbertrand oh yeah. Wasn't working for me either, So as a workaround, I returned some dummy values from the function so that the command could succeed without Jomba API error. The Jobma API functionality isn't integral part of this PR or it's scope so I think placeholder values should be fine.

I changed the create_interview_in_jobma to be:

def create_interview_in_jobma(interview):
    """
    Create a new interview on Jobma

    Args:
        interview (Interview): An interview object
    """
    client = get_jobma_client()
    url = urljoin(settings.JOBMA_BASE_URL, "interviews")
    job = interview.job
    first_name, last_name = get_first_and_last_names(interview.applicant)
    # response = client.post(
    #     url,
    #     json={
    #         "interview_template_id": str(job.interview_template_id),
    #         "job_id": str(job.job_id),
    #         "job_code": job.job_code,
    #         "job_title": job.job_title,
    #         "callback_url": urljoin(
    #             settings.SITE_BASE_URL,
    #             reverse("jobma-webhook", kwargs={"pk": interview.id}),
    #         ),
    #         "candidate": {
    #             "first_name": first_name,
    #             "last_name": last_name,
    #             "phone": "",
    #             "email": interview.applicant.email,
    #         },
    #     },
    # )
    # response.raise_for_status()
    # result = response.json()
    interview_link = "https://test_link.casas"
    if interview_link is not None:
        interview.interview_url = interview_link
    else:
        log.error("Interview link not found in payload - %s", result)

    interview_token = "11212"
    if interview_token is not None:
        interview.interview_token = interview_token

    interview.save_and_log(None)
    return interview_link

@mbertrand Do you think we should test it otherwise?

[mbertrand]

Modified code to bypass the 500 error from jobma, on first run of the following everything went according to plan:

mitodl@fe24a0c0088d:/src$ python manage.py set_application_state --app-id=1 --state=AWAITING_PAYMENT
Application: 
user='mrbertrand@gmail.com', run='Seed Single Variable Calculus Run 2', state=AWAITING_PAYMENT
State change: AWAITING_RESUME -> AWAITING_PAYMENT

mitodl@fe24a0c0088d:/src$ python manage.py reset_application_interview_state --user=2 --run=2
Jobma interview state has been reset for User=matthew-r-bertrand Run=Seed Single Variable Calculus, Sep 11 - Dec 10, 2023.

mitodl@fe24a0c0088d:/src$ python manage.py reset_application_interview_state --user=2 --run=2
CommandError: User's application is not in a reviewable state. User=matthew-r-bertrand, Run=Seed Single Variable Calculus, Sep 11 - Dec 10, 2023, State=AWAITING_USER_SUBMISSIONS.

But then I ran the following again:

mitodl@fe24a0c0088d:/src$ python manage.py set_application_state --app-id=1 --state=AWAITING_PAYMENT
Application: 
user='mrbertrand@gmail.com', run='Seed Single Variable Calculus Run 2', state=AWAITING_PAYMENT
State change: AWAITING_USER_SUBMISSIONS -> AWAITING_PAYMENT

and the frontend indicated that the application was awaiting payment but the interview step was still awaiting completion:

[mbertrand]

@arslanashraf7 I followed your approach to test locally. However it appears interviews are still broken on RC, which will make it difficult to test there. I tried to fix it awhile back but didn't have much luck and moved on to other things. @pdpinch should I try to fix this again now?

[arslanashraf7]

But then I ran the following again:

mitodl@fe24a0c0088d:/src$ python manage.py set_application_state --app-id=1 --state=AWAITING_PAYMENT
Application: 
user='mrbertrand@gmail.com', run='Seed Single Variable Calculus Run 2', state=AWAITING_PAYMENT
State change: AWAITING_USER_SUBMISSIONS -> AWAITING_PAYMENT

@mbertrand That might probably be because your application step submission's Submission Status would be in Pending state, So the step exists but the submission status is pending that's why It doesn't show the green timeline for this. This might be something related to how the set_application_state command is working.

[mbertrand]

Working with JOBMA_BASE_URL=https://dev.jobma.com (though interview link redirects to an "Interview deleted" page), so I think that should be good enough for RC testing.