Regarding model training

[xpc7-lite]

Hello, author. I'm wondering if I want to train a model based on the LIVE-VQC video dataset on my own, and also pre-train the spatial model by myself (as described in your paper), in what order should I run your code and how should I set the parameters?
Previously, I first used extract_features.py to extract the features of the LIVE dataset and KonIQ. Then I ran spatial_train.py to pre-train the spatial model using the features of KonIQ. After that, I ran end2end_train.py to train the model for the entire LIVE dataset using the features of the LIVE data and the pre-trained spatial model. However, when running test_model.py, I found that all the performance metrics were lower than those reported in your paper, and I also got the following error:
RuntimeWarning: overflow encountered in exp
logisticPart = 1 + np.exp(np.negative(np.divide(X - bayta3, np.abs(bayta4))))

[Ahmed-Telili]

Hello,

Thank you for reaching out. As you mentioned, the correct order is to first pre-train the spatial model using spatial_train.py. I recommend verifying that your pre-trained spatial model achieves performance comparable to the one provided in /models before proceeding. Next, you should run end2end_train.py.

Hyperparameter settings:

• Overlapping: 0.2
• Number of frames: 30
• Number of patches: 25

Additionally, please note that dataset modifications, such as adding or removing videos, can also impact performance.
Feel free to try these steps, and let me know if you encounter any further issues.

[xpc7-lite]

So, do I not need to use extract_features.py to extract features? The features I had previously were extracted from the features of the live-VQC dataset downloaded from the official website using extract_features.py. What features should I use now to train the spatial model?

[xpc7-lite]

Could you please tell me the detailed steps to train a model using the live-vqc dataset from the official website and then test it?

[Ahmed-Telili]

For sure you need to extract features using extract_features.py

Steps to Train the Model:

1. Feature Extraction: Run extract_features.py to extract the necessary features.
2. Spatial Model Training: Use spatial_train.py to train the spatial model.
3. End-to-End Training: Finally, train the entire model by running end2end_train.py.

[xpc7-lite]

This is exactly how I trained the models. But why does the training of both the spatial model and the final model always end prematurely? Moreover, the Mean Opinion Score (MOS) predicted by the trained models is far greater than the range of 0 - 100 in the training set, reaching around 200 - 300. Additionally, when using test_model.py for testing, the index is abnormal. Here are my detailed steps:
I used extract_features.py to extract the features of the live - vqc video dataset (for training the final model) and the features of the koniq - 10k image dataset (for training the spatial model) respectively.
I ran spatial_train.py to train the spatial model using the features of the image dataset extracted just now (the whole model was saved).
I ran end2end_train.py to train the final model by combining the trained spatial model from step 2 and the video features extracted in step 1 (the model weights were saved).
I ran test_model.py. I modified the default models in the code when dataset = 'live' to the spatial model (the whole model) and the final model (the model weights) trained in the above steps. When I ran test_model.py --dataset live, the above - mentioned problems occurred.