Line search is stuck

[rs1909]

I am trying to optimise a rather complex problem, which is a direct product of many Stiefel manifolds of various sizes and also some Euclidean using quasi_Newton.

The algorithm get always stuck after little progress is being made. For example

Initial | F(x): 217204.0892916842 | |gradF(x)|:1.411402378033978e6 | 
# 1 | F(x): 125255.998499018 | |gradF(x)|:1.3457289674519533e6 | 
# 2 | F(x): 82572.08843396472 | |gradF(x)|:16138.519481929883 | 
# 3 | F(x): 45896.72961391446 | |gradF(x)|:13171.296544122759 | 
# 4 | F(x): 21290.493465918968 | |gradF(x)|:11204.09014255528 | 
# 5 | F(x): 3373.4371039798216 | |gradF(x)|:5854.715010866772 | 
# 6 | F(x): 540.6498309427075 | |gradF(x)|:2377.0557594470984 | 
# 7 | F(x): 93.71939815247248 | |gradF(x)|:818.1653565168624 | 
# 8 | F(x): 39.85831181730066 | |gradF(x)|:417.27137524875155 | 
# 9 | F(x): 31.686480595334665 | |gradF(x)|:201.9046046349952 | 
# 10 | F(x): 27.45855512651261 | |gradF(x)|:108.37829454859812 | 
# 11 | F(x): 27.208921309967984 | |gradF(x)|:67.68750304653418 | 
# 12 | F(x): 26.266898564398325 | |gradF(x)|:106.00785004358552 | 
# 13 | F(x): 26.266898565320133 | |gradF(x)|:77.08155071828511 | 
# 14 | F(x): 26.26689855785398 | |gradF(x)|:77.08155072605182 | 
# 15 | F(x): 26.266898557853978 | |gradF(x)|:77.08155057576158 | 
# 16 | F(x): 26.26686389923603 | |gradF(x)|:77.08155058031262 | 
# 17 | F(x): 26.266863899237364 | |gradF(x)|:77.08116106600322 | 
# 18 | F(x): 26.21433916910066 | |gradF(x)|:77.08116829017843 | 
# 19 | F(x): 26.17871121677865 | |gradF(x)|:76.4778899019015 | 
# 20 | F(x): 24.398585657373843 | |gradF(x)|:55.456392272502875 | 
# 21 | F(x): 24.006019557512417 | |gradF(x)|:45.59338716612488 | 
# 22 | F(x): 23.974285526631956 | |gradF(x)|:33.996491209448344 | 
# 23 | F(x): 23.929038306658274 | |gradF(x)|:30.705556622577866 | 
# 24 | F(x): 23.92903830674107 | |gradF(x)|:29.61783626646946 | 

I was going through the documentation and setting various parameters for the line search algorithms, but nothing helps, if anything it just gets worse, i.e., gets stuck earlier.
Is there any advice on how to avoid this?

Thanks

[kellertuer]

Without any example code, it is not possible for me to help out much but just provide generic advice:

• what usually helps but slows down everything (in time) is to set the decrease factor to 0.999 in https://manoptjl.org/stable/plans/index.html#Manopt.ArmijoLinesearch
• What does the step size do? I fear it looks like it is basically zero after step 24 or something?
• You could try a constant or decreasing step size first to get some intuition about step sizes that work well (i.e. where you have a decrease in the function value and no increase
• Generally Armijo might get to too small step sizes, you can also try to use https://manoptjl.org/stable/plans/index.html#Manopt.WolfePowellLineseach