forked from LCDsoft/ILCDIRAC
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathrelease.notes
2129 lines (1421 loc) · 52.2 KB
/
release.notes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
[v26r0p14]
*Interfaces
NEW: (!150) :class:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.Whizard2.Whizard2` interface (ILCDIRAC-467)
[v26r0p13]
*Interface
NEW: (!153) `ILCDIRAC.Interfaces.API.NewInterface.UserJob.UserJob.setSplitDoNotAlterOutputFilename`: function to disable the injecting of the JobIndex into the filename
*ILCTransformationSystem
FIX: (!155) DataRecoveryAgent: fix problem to get canceled request status. Fixes problem were the request cannot be successful, and thus some jobs were not dealt with properly.
FIX: (!156) `dirac-clic-make-productions` use ProdID as string, fixes problem if ProdID is 0
CHANGE: (!156) `dirac-clic-make-productions` *_productionname* function no longer has *prodname* argument
*Interfaces
CHANGE: (!156) Updated docstring for :func:`~ILCDIRAC.Interfaces.Utilities.DDInterfaceMixin.DDInterfaceMixin.setDetectorModel` to include examples of the different options
*Workflow
NEW: (!153) if some kind of jobSplitting is done the OutputData LFNs are automatically injected with the JobIndex in the splitting sequence. See :mod:`ILCDIRAC.Core.Utilities.Splitting`
[v26r0p12]
*Core
CHANGE: (!141) **GetOverlayFiles** module renamed to :mod:`~ILCDIRAC.Core.Utilities.OverlayFiles` and added some helper functions to it; reduce code duplication in the OverlayInput interface and workflow module
CHANGE: (!151) `dirac-ilc-add-cvmfs-software` add dryrun option, add possibility to add any Config package
*Workflow
NEW: (!153) if some kind of jobSplitting is done the OutputData LFNs are automatically injected with the JobIndex in the splitting sequence. See :mod:`ILCDIRAC.Core.Utilities.Splitting`
*ILCTransformationSystem
FIX: (!140) :mod:`~ILCDIRAC.ILCTransformationSystem.Utilities.MovingTransformation`: fix the *targetSE* argument type; fix if *targetSE* is a string
FIX: (!140) `dirac-clic-make-productions`: *Move* parameter was not parsed before, now moving transformations are also created if it is True
FIX: (!144) `dirac-clic-make-productions` make more parameters optional
CHANGE: (!144) `dirac-clic-make-productions` change parameter name *NumberOfEventsInBaseFiles* to *eventsInSplitFiles*
CHANGE: (!144) `dirac-clic-make-productions` change parameter *overlayEnergy* to *overlayEvents*
CHANGE: (!144) `dirac-clic-make-productions` Produciton Group is now taking only from the config file, no additional string is added. Use string replacements for automatic group names
FIX: (!152) `dirac-clic-make-productions`: allow moving of DST files
NEW: (!152) `dirac-clic-make-productions`: add cliReco option to set extraCLIArguments for Marlin
*Docs
CHANGE: (!140) move production manager notes from the twiki to the ilcdirac docs
NEW: (!140) CLIC Production manager guide with some notes for CLIC productions
CHANGE: (!142) restructuring of the documentation; additional example script
NEW: (!147) Add JIRA issue collector to the doc pages.
*Interface
NEW: (!153) :func:`~ILCDIRAC.Interfaces.API.NewInterface.UserJob.UserJob.setSplitDoNotAlterOutputFilename`: function to disable the injecting of the JobIndex into the filename
NEW: (!146) :class:`~ILCDIRAC.Interfaces.API.NewInterface.UserJob.UserJob`: new functions to ease the automatic splitting of jobs :func:`~ILCDIRAC.Interfaces.API.NewInterface.UserJob.UserJob.setSplitInputData`, :func:`~ILCDIRAC.Interfaces.API.NewInterface.UserJob.UserJob.setSplitEvents`
[v26r0p11]
*Interface
NEW: (!136) new attribute *processorName* for :class:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.OverlayInput.OverlayInput`. Set the name of the Marlin processor the input files belong to. This enables multiple use of the OverlayInput application to download files for different Marlin processors.
CHANGE: (!136) :class:`~ILCDIRAC.Interfaces.API.NewInterface.ProductionJob.ProductionJob` change capitalisation of the *version* argument of :func:`~ILCDIRAC.Interfaces.API.NewInterface.ProductionJob.ProductionJob.setConfig`
*ILCTransformationSystem
NEW: (!128) `dirac-clic-make-productions` script to create productions for the new ddsim/marlin software chain
NEW: (!124) `dirac-ilc-get-prod-log` new options to get all files in a folder (-A) and to automatically start downloading without confirmation (-N)
NEW: (!126) `dirac-ilc-moving-transformation`: new option (--allFor) to set a prodID and have the three corresponding moving transformations: prodID/GEN, prodID+1/SIM and prodID+2/REC created. Added consistency check for Datatype
*Interfaces
NEW: (!125) :class:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.StdHepSplit.StdHepSplit`: new property to set maxRead flag for the hepsplit application
*Workflow
CHANGE: (!135) The output parser for :class:`~ILCDIRAC.Workflow.Modules.MarlinAnalysis.MarlinAnalysis` will now pass lines containing 'EventNumber' or 'StatusMonitor' to the standard output, in addition to 'event' and 'ProgressHandler'
CHANGE: (!136) :class:`~ILCDIRAC.Workflow.Modules.OverlayInput.OverlayInput` fix the path for /eos/experiment/clicdp to get files directly from EOS
CHANGE: (!136) :class:`~ILCDIRAC.Workflow.Modules.MarlinAnalysis` can now handle setting of parameters for Overlay processors defined in a processor group. Implemented in the new module `Core.Utilities.MarlinXML`
NEW: (!125) :class:`~ILCDIRAC.Workflow.Modules.StdHepSplit.StdHepSplit`: set "--maxread" if maxRead is set for the application, no changes if maxRead is 0 for backward compatibility
CHANGE: (!127) Marlin the DST/REC files for processors called Output_DST/Output_REC files in production environments are now handled as well. For Users all processors of Type LCIOOutputProcessor should be used to change the output file name, not just the one called 'MyLCIOOutputProcessor'
*Docs
NEW: (!130) import documentation from ILCDIRAC-Ops repo, always compile docs
NEW: (!137) *docs/GetReleaseNotes.py* script to get the release notes from the merged merge requests since given date
[v26r0p10]
* Interfaces:
CHANGE: DDSim: use *detectorModel* as the *detectorType*, needed in CLIC productions
NEW ProctionJob: add :func:`~ILCDIRAC.Interfaces.API.NewInterface.UserJob.UserJob.setCLICConfig` function to add ClicConfigPackage to productions
* Core:
CHANGE: :mod:`~ILCDIRAC.Core.Utilities.InstalledFiles`: allow using arbitrary ConfigPackage to provide steering files. For now ILDConfig or ClicConfig defined in ConfigurationSystem
* Workflow:
NEW: :class:`~ILCDIRAC.Workflow.Modules.ModuleBase.ModuleBase`: General :func:`~ILCDIRAC.Workflow.Modules.ModuleBase.ModuleBase.treatConfigPackage` function to replace treatILDConfigPackage backward compatible
[v26r0p9]
* Workflow:
NEW: :class:`~ILCDIRAC.Workflow.Modules.SLICAnalysis`: allow user to preload libraries for SLIC. Add a tarball with folder 'preload' to the Sandbox. Libraries (files containing '.so' in that folder will be added to the LD_PRELOAD environment variable.
[v26r0p8]
* Core:
NEW: Based on v6r15p27
NEW: ILD FilenameEncoder now also hadles files for Split transformations
* ILCTransformationSystem
FIX: DataRecoveryAgent: No longer send messages if nothing changed.
FIX: DataRecoveryAgent: Deal with MCSimulation/MCReconstruction jobs where the inputFile does not exist
* Interfaces
FIX: Changes to ILDProductionChainOpt2017 and DBD: Define output SEs, limit number of jobs for all productions
[v26r0p7]
* Workflow
NEW: DDSim: ddsim now allows a user to add their own plugins.
A tarball with the content lib/libmydd4hepplugin.so, lib/libmydd4hepplugin.components can be added to the input sandbox
NEW: Marlin: Marlin also can use the dd4hep plugins provided by a users
FIX: detector model zipfiles/tarballs for marlin were not working before
[v26r0p6]
* Core
NEW: FilenameEncoder by Akiya Miyamoto
CHANGE: OutputDataPolicy: "\*_ILD" production types use FilenameEncoder
* Interfaces:
CHANGE: Rename ILDProductionJob to ILDProductionJobDBD for DBD productions
CHANGE: Rename ILDProductionChain to ILDProductionChainDBD
NEW: ILDProductionJobOpt2017: Production job class using the FilenameEncoder
NEW: ILDProductionChainOpt2017: using ILDProductionJobOpt2017
* ILCTransformationSystem:
FIX: DataRecoveryAgent: Fixed a bug that caused files to be reset too often
* Workflow:
CHANGE: DDSim Workflow module is now picking up ProductionOutputdata from WorkflowCommons instead of re-creating the filename
[v26r0p5]
* Core:
NEW: Based on DIRAC v6r15p25
*Interfaces:
NEW: More tests for interface modules
[v26r0p4]
* Core:
NEW: Based on DIRAC v6r15p23
* Interfaces
CHANGE: The Marlin application no longer has a default GearFile because having a gear file causes problems when dd4hep geometry is used instead.
* DataManagement:
CHANGE: (from DIRAC): dirac-dms-find-lfns command signature changed, now the PATH argument has to be used to define the basepath for the search
* ILCTransformation
FIX: dirac-ilc-production-summary: fix bug causing a crash when whizard details did not exist
[v26r0p3]
* Core:
NEW: Based on DIRAC v6r15p17
FIX: ILCDIRAC-718 pilots at KEK not matching jobs
FIX: ILCDIRAC-713 dirac-dms-find-lfns produced output files not readable by other dirac commands
NEW: Tests also running on Centos7 now
* ILCTransformationSystem:
NEW: plugin BroadcastProcessed to create transformations to move only files with descendents
NEW: dirac-ilc-moving-transformation creates BroadcastProcessed transformations for GEN/SIM files
* Workflow:
FIX: Hopefully clearer message when packages are not found directly on CVMFS, ILCDIRAC-708
[v26r0p2]
* Core:
NEW: More Unite tests
NEW: Based on DIRAC v6r15p12 (-alpha?) Fixe for ILCDIRAC-703
FIX: `dirac-ilc-list-users` needed to be adapted for interface change in :mod:`DIRAC.Core.Security.VOMSService`, ILCDIRAC-702
* ILCTransformationSystem:
FIX: DataRecoveryAgent using DiraAPI getJobJDL function to get dictionary of JDLs, ILCDIRAC-705
* Interfaces:
FIX: Fix exception in internal :meth:`DIRAC.Core.Base.API.API._reportError` function when function argument is 'name'
[v26r0p1]
* Core:
NEW: Based on DIRAC v6r15p11 (-alpha) to solve a problem with accessing files on RAL
[v26r0]
* Core:
NEW: Based on DIRAC v6r15p10
NEW: Many unittests for Utilities and other modules
CHANGE: Removed obsolete modules
* Workflow:
FIX: DDSim randomSeed will be set to the jobID by default
* Interfaces:
FIX: ILDProductionJob code cleaned and some bugs fixed, ILCDIRAC-641
FIX: Some more errors from application configurations will now be reported before submission
* Utilities:
FIX: Clean code in CombinedSoftwareInstallation. Unnecessary call to os.path.join.
FIX: PrepareLibs refactored
FIX: PrepareOption files refactored for better maintanability
* Resources:
CHANGE: Go back to XROOTStorage from DIRAC
* ILCTransformationSystem:
NEW: Add script to create "moving" transformations. Moving files from one SE to another (replicate and the remove replica)
[v25r0p10]
* Core:
NEW: based on DIRAC v6r14p37
FIX: Add CVMFS Software script now adds DDSimDetectorModels
[v25r0p9]
* ILCTransformationSystem:
FIX: ILCDIRAC-665: Also handle cases if a file gets stuck in 'Processed' status
* Overlay:
NEW: Set the NSKipEventsRandom parameter for BGOverlay processor. Calculates number of Events based on NBEvts Configuration Parameter
NEW: Allow defining different parameter for the same background type by using EventType parameter to set the correct EventType
* Interfaces:
NEW: When applications check for existance of steering files packages on CVMFS are considered as well
* Workflow:
NEW: Move back to CPUNormalisation from DIRAC, fix ILCDIRAC-661 implemented and released in DIRAC
[v25r0p8]
* Core:
NEW: Based on DIRAC v6r14p35
* Workflow:
FIX: :mod:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.DDSim`: Fix for running ddsim when init script is needed. The LD_LIBRARY_PATH was not put together correctly and on some sites the local libstdc++.so was picked up.
[v25r0p7]
* Workflow:
FIX: :mod:`~ILCDIRAC.Workflow.Modules.OverlayInput` for KEK was not getting the correct path to copy files.
FIX: :mod:`~ILCDIRAC.Workflow.Modules.DDSimAnalysis`: Enable use of tarballs with non-system compiler, ILCDIRAC-671
* Interface:
NEW: :mod:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.Marlin` has the new function :func:`~ILCDIRAC.Interfaces.Utilities.DDInterfaceMixin.DDInterfaceMixin.setDetectorModel` that works the same way as the :mod:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.DDSim` function and can be used to set the dd4hep geometry based model. (ILCDIRAC-659)
[v25r0p6]
* Workflow:
FIX: :mod:`~ILCDIRAC.Workflow.Modules.MokkaAnalysis` outputfile event parameters were not properly set for Polarisation
CHANGE: :mod:`~ILCDIRAC.Workflow.Modules.OverlayInput` files are taken from cern eos instead of castor, when running at cern (ILCDIRAC-636)
* Interfaces:
FIX: Files from the ParametricInputData should now be handled correctly
[v25r0p5]
* WorkloadManagement:
FIX: Copy :func:`getCPUNormalization` from DIRAC and reduce memory footprint ILCDIRAC-661
[v25r0p4]
* WorkloadManagement:
FIX: Copy :mod:`DownloadInputData` from DIRAC and fix ILCDIRAC-655
[v25r0p3]
* Core:
NEW: Based on dirac v6r14p26
* ILCTransformationSystem:
FIX: :mod:`~ILCDIRAC.ILCTransformationSystem.Agent.DataRecoveryAgent` now properly sets MaxReset for files with a large ErrorCount
* Workflow:
FIX: :mod:`~ILCDIRAC.Workflow.Modules.DDSimAnalysis` Add DD4hepInstall environment variable to script for TarBallBased installations. Fixes problem for running these jobs locally.
[v25r0p2]
* Core:
NEW: Based on dirac v6r14p24
* Workflow:
FIX: :mod:`~ILCDIRAC.Workflow.Modules.DDSimAnalysis`: correctly handle the tarball
[v25r0p1]
* Interfaces:
FIX: Correctly handle :mod:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.DDSim` detector models
[v25r0p0]
*Core
NEW: These release notes are the summary of every pre-release after the last v23r1 release
NEW: Based on dirac v6r14p18, many changes in Dirac
*Interfaces
NEW: Added :mod:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.DDSim` Interfaces, ILCDIRAC-507
*Workflow
NEW: Added :mod:`~ILCDIRAC.Workflow.Modules.DDSimAnalysis` Workflow Module, ILCDIRAC-507
*ILCTransformation
FIX: Adapt to interface changes in DIRAC/Transformation, ILCDIRAC-599
[v25-pre3]
*Interfaces
NEW: Added :mod:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.DDSim` Interfaces, ILCDIRAC-507
*Workflow
NEW: Added :mod:`~ILCDIRAC.Workflow.Modules.DDSimAnalysis` Workflow Module, ILCDIRAC-507
[v25-pre2]
*Core
NEW: Based on Dirac v6r14p11
FIX: XRootStorage is working again thanks to v6r14p20
CHANGE: Removed some deprecated and unused packages
NEW: Added many tests and made them compatible with nosetests
[v25-pre1]
*Core
NEW: Based on Dirac v6r14p10
*ILCTransformation
FIX: Adapt to interface changes in DIRAC/Transformation, ILCDIRAC-599
[v24-pre5]
*Core
NEW: Based on Dirac v6r13p14
[v24-pre3]
*Core
NEW: Based on Dirac v6r13p9
[v24-pre2]
*Core
NEW: Based on Dirac v6r13p8
[v24-pre1]
*Core
NEW: Based on Dirac v6r13p7
############################################################
[v23r1p1]
*Note
CHANGE: Moved many scripts to different locations (ILCDIRAC-581) for easier
automatic generation of the documentation. This requires a re-installation of
the ILCDIRAC client to remove outdated files. Apart from this
*Interfaces
CHANGE: Removed RemoveFileAndDaughtersWhenFileHasNoAncestor.py, script superseded by one in ILCDIRAC-Ops and :mod:`~ILCDIRAC.ILCTransformationSystem.Agent.DataRecoveryAgent`
CHANGE: Moved :any:`dirac-ilc-get-info` to `ILCTransformationSystem <ILCDIRAC.ILCTransformationSystem.scripts>`
CHANGE: Moved `dirac-ilc-get-prod-log` to `ILCTransformationSystem <ILCDIRAC.ILCTransformationSystem.scripts>`
CHANGE: Moved `dirac-ilc-production-summary` to `ILCTransformationSystem <ILCDIRAC.ILCTransformationSystem.scripts>`
CHANGE: Moved `dirac-ilc-replication-transformation` to `ILCTransformationSystem <ILCDIRAC.ILCTransformationSystem.scripts>`
CHANGE: Moved `dirac-ilc-upload-gen-files` to `ILCTransformationSystem <ILCDIRAC.ILCTransformationSystem.scripts>`
FIX: Updated :mod:`~ILCDIRAC.Interfaces.API.NewInterface.Applications.Marlin` documentation
FIX: Fix a typo `UserJob.setInputSandbox` documentation
*Core
CHANGE: Moved `ilcdirac-version` to `Interfaces <ILCDIRAC.Interfaces.scripts>`
CHANGE: Moved `dirac-ilc-show-software` to `Interfaces <ILCDIRAC.Interfaces.scripts>`
CHANGE: The log file failover location was changed so that log files are not picked up by the InputDataAgent for productions
################################################################################
[v23r0p12]
*Workflow
FIX: ILCDIRAC-612: Watchdog was not disabled properly when downloading overlay files
FIX: ILCDIRAC-612: Logging output was wrong when copying overlay files at IMPERIAL
[v23r0p11]
Based von DIRAC v6r12p53, see also v6r12p52
*Workflow
FIX: :mod:`~ILCDIRAC.Workflow.Modules.RootMacro` and :mod:`~ILCDIRAC.Workflow.Modules.RootScript` can now use the software versions natively installed on CVMFS (ILCDIRAC-582)
FIX: :mod:`~ILCDIRAC.Workflow.Modules.RootScript` can now also be used to run binaries located in the PATH, e.g., 'hadd' to join root files
*ILCTransformationSystem
NEW: :any:`dirac-ilc-get-prod-log` now accepts -P prodID to get log files for given productionID (C. Calancha, ILCDIRAC-585)
FIX: :mod:`~ILCDIRAC.ILCTransformationSystem.Agent.DataRecoveryAgent` now ignores productions where nothing has changed since the last cycle, (ILCDIRAC-588)
[v23r0p10]
Based on DIRAC v6r12p51
*WorkloadManagement
NEW: :mod:`~ILCDIRAC.WorkloadManagementSystem.Executor.SoftwareVersions` Executor: Takes care to ban sites which are not compatible with the SofwarePackages required for any given job
*ILCTransformation
FIX: :mod:`~ILCDIRAC.ILCTransformationSystem.Agent.DataRecoveryAgent`: Fixes for notification, updates configuration before every cycle
*RequestManagement
FIX: :mod:`~ILCDIRAC.RequestManagementSystem.Agent.RestartReqExeAgent` deal with multiple PIDs
[v23r0p9]
*Core
NEW: Based on Dirac v6r12p50
*Interfaces
FIX: DIRAC FileCatalog CLI: fixes for path arguments
[v23r0p8]
*Workflow
FIX: Set OMP_NUM_THREADS=1 for all in :mod:`~ILCDIRAC.Workflow.Modules.ModuleBase` and in the execution of the Script in :mod:`~ILCDIRAC.Workflow.Modules.ApplicationScript`
*ILCTransformation
NEW: Rewrite of the :mod:`~ILCDIRAC.ILCTransformationSystem.Agent.DataRecoveryAgent`
[v23r0p6]
*Interfaces
FIX: :any:`dirac-ilc-upload-gen-files`: ILCDIRAC-571 correct energy variable, better logging for metadata setting in case of errors
*RequestManagement
NEW: :mod:`~ILCDIRAC.RequestManagementSystem.Agent.RestartReqExeAgent` to restart the RequestManagementAgent when it gets stuck, ILCDIRAC-572
*Workflow
FIX: ILCDIRAC-576: :mod:`~ILCDIRAC.Workflow.Modules.ApplicationScript`: Set OMP_NUM_THREADS=1
[v23r0p5]
*Core
NEW: Option to set the location of the shared area, needed to run with OSG cvmfs for now
[v23r0p4]
*Core
FIX: Based on Dirac v6r12p49, for real this time
NEW: ilcdirac-version script to display the ilcdirac and dirac versions
[v23r0p3]
*Core
FIX: Based on Dirac v6r12p48
NEW: createTarBallForProgram.sh script to create a tarball of the executable and required libraries
*Interfaces
FIX: ProductionJob: Adapt to new changePathMode interface (ILCDIRAC-544)
FIX: dirac-ilc-add-user: Adapt to new changePathOwner, changePathGroup interface
FIX: dirac-ilc-replication-transformation: check that ilc_prod proxy is used
[v23r0p2]
*Core
FIX: Based on Dirac v6r12p47, fixes issue with removing metadata in the filecatalog
*Interfaces
FIX: ILCDIRAC-547: Give error when dirac-ilc-find-in-FC does not get a path as first argument
[v23r0p1]
*Core
NEW: based on Dirac v6r12p46: Many fixes for Xroot(EOS), GlobusCEs
*Interfaces
FIX: ILCDIRAC-537: ILDProductionChain set different basepath for stdhepsplit productions
[v23r0]
*Core
NEW: Based on Dirac v6r12p44
*Interfaces
FIX: Cleanup of interfaces member variables, should be completely transparent for users
*Transformation
FIX: Fixes for ILDProductionJob from C. Calancha (ILCDIRAC-413)
[v23-pre5]
*Core
NEW: Based on Dirac v6r12p44
*Interfaces
FIX: Cleanup of interfaces member variables, should be completely transparent for users
[v23-pre3]
*Core
NEW: Based on Dirac v6r12p42
*Transformation
FIX: Fixes for ILDProductionJob from C. Calancha (ILCDIRAC-413)
[v23-pre1]
*Core
NEW: Based on Dirac v6r12-pre21: DO NOT USE
[v22r2p5]
*Transformation
NEW: Script to create replication transformations dirac-ilc-replication-transformation
*Workflow
FIX: OverlayInput will only waste time after downloading the first file
*Dirac
NEW: based on v6r11p5 (GlobusComputingElement)
[v22r2p4]
*Workflow
FIX: Fix if ILDConfig cannot access CVMFS
[v22r2p3]
*Dirac
NEW: Based on Dirac version v6r11p33: Issue regarding space in CE_OS description
*Workflow
FIX: ILCDIRAC-413 fix regarding picking up ILDConfig from CVMFS
*RMS
FIX: LogUpload would place log files in the wrong location
*Scripts
FIX: ILCDIRAC-474: dirac-ilc-add-whizard fixes for creating the replication request
*Core
FIX: ILCDIRAC-479: Fix python warning
[v22r2p2]
*Dirac
NEW: Based on Dirac version v6r11p32: Fixes issue for dirac-ilc-list-user
[v22r2p1]
*Workflow
FIX: OverlayInput will now always try to get files from the DataManager if files do not exist on local SE
[v22r2p0]
*DIRAC
NEW: v6r11p31
*Workflow
CHANGE: Tar and compress production logfiles
FIX: CVMFS support for native installations (Mokka, Marlin)
FIX: Fix some of the issues with module- and step_commons-parameters
FIX: Mokka: ProcessID is now passed to the module
FIX: Remove SystemConfig/sysconfig, replace with platform
FIX: Bugfixes/conventions/pylint cleanup, refactoring
[v22r2-pre4]
*Workflow
FIX: Fix some of the issues with module and step_commons - parameters
FIX: Mokka: ProcessID is now passed to the module
[v22r2-pre3]
*DIRAC
NEW: v6r11p31
*Workflow
FIX: CVMFS support for native installations (Mokka, Marlin)
FIX: Bugfixes/conventions/pylint cleanup, refactoring
[v22r2-pre1]
*Workflow
CHANGE: Tar and compress logfiles
FIX: Remove SystemConfig/sysconfig, replace with platform
[v22r1p15]
*Workflow
FIX: ILCDIRAC-452: getNumberOfEvents will now fail if no information is found. if NumberOfEvents is 0 the job will fail.
[v22r1p14]
*Workflow
FIX: MokkaAnalysis: wait 100 seconds for socket
FIX: CombinedSoftwareInstallation: Allow for mixing sharedarea (CVMFS) and LocalArea software
[v22r1p13]
*Workflow
FIX: MokkaAnalysis: Place mysql logs in subdirectory $CWD/mysqllogs
[v22r1p12]
*Workflow
FIX: ILCDIRAC-442: We need to run removeLibc correctly, pointing to the library folder
[v22r1p11]
*Workflow
FIX: ILCDIRAC-442: We need to run removeLibc in case mysql4grid is not on cvmfs
[v22r1p10]
*Workflow
FIX: ILCDIRAC-442: mysql.sock must be in /tmp folder
FIX: ILCDIRAC-442: Don't copy mysql4grid
[v22r1p9]
*Workflow
FIX: ILCDIRAC-442: copy mysql4grid to local working directory
[v22r1p8]
*Workflow
FIX: ILCDIRAC-442: more debugging output
[v22r1p7]
*Workflow
FIX: ILCDIRAC-302: SlicPandora would remove detector.zip: fix exception in unzip function that caused this issue
FIX: MarlinAnalysis: Fix Issue when LCFIVertex and LCFIPlus are both in the MARLIN_DLL
FIX: ILCDIRAC-442: MokkaAnalysis: Fix when mysql is available via CVMFS
[v22r1p6]
*General
NEW: Dirac version v6r11p31
*FileCatalog
NEW: FC Search: new flag -D to only display folders
*Interfaces
CHANGE: use ilcdirac-support@cern.ch for email address
FIX: ProductionJob ILCDIRAC-427: Register energy metadata
FIX: ProductionJob ILCDIRAC-427: Accept energy metadata as integer
*Core
CHANGE: use ilcdirac-support@cern.ch for email address
CHANGE: Allow specifying additional libraries to remove in removeLibc function
*ILCTransformation
FIX: DataRecoveryAgent: Use the new ReqClient
*Workflow
NEW: Mokka: Print libraries used by Mokka and mysql executable (debug only)
FIX: ILCDIRAC-423: remove libgcc_s.so.1 from older mysq4grid versions
[v22r1p5]
*Workflow
FIX: ILCDIRAC-422: Mokka did not work when ILDConfig parameter set
[v22r1p4]
*General
NEW: Dirac version v6r11p28
*Transformation
FIX: TransformationCleaningAgent will now cleanup files when cleaning productions
*Workflow
FIX: Print correct message about files being uploaded
[v22r1p3]
*Workflow
FIX: ILCDIRAC-417: Missing Checksum entry when creating RegistrationRequests during Failover
FIX: ILCDIRAC-418: Properly cleanup requests when production jobs fail and we want to remove possible output files
[v22r1p2]
*Workflow
FIX: ILCDIRAC-406: Create removal requests for production output when any kind of failure happens
FIX: OverlayInput: only print list of files for DEBUG
FIX: UserJobFinalization: Actually print the content of working directory as promised
*Interfaces
CHANGE: Move application to their own file, transparent change for users
[v22r1p1]
*Workflow
FIX: ILCDIRAC-400 Backward compatibility for old jobs/productions still using SystemConfig
*ILCTransformationSystem
FIX: ILCDIRAC-399 ILCTransformationAgent: Adapt to DataManager
FIX: ILCDIRAC-398 DataRecoveryAgent: Fixed SyntaxError and KeyError
[v22r1p0]
*Core
CHANGE: Tag for official first release based on v6r11p25 Contains all the changes for v22r0
[v22r0p10]
*Workflow
NEW: ILCDIRAC-222: Print working directory content also for user jobs. Print content recursively
NEW: ILCDIRAC-381: OverlayInput: One can now set the path at which the Overlay files should be picked up. This ignores all of the meta data
*Interfaces
NEW: ILCDIRAC-381: OverlayInput: new function setPathToFiles to set the path where the overlay files should be picked up. There is a check that there are files in that folder, but there is no check if it makes sense to overlay these files.
NEW: OverlayInput: Two alternative spellings for old functions: setNumberOfSignalEventsPerJob and setBackgroundType, alternatives for setNbSigEvtsPerJob and setBkgEvtType, respectively.
[v22r0p9]
*Workflow
FIX: ILCDIRAC-384: Do not submit failover to same SE as targetSE
FIX: ILCDIRAC-382: Allow job.setExecutable to be run before Workflow modules
[v22r0p8]
Workflow
*CHANGE: Change the way removal request is created
[v22r0p7]
*Workflow
FIX problems for request
FIX: ModuleBase: getRequestContainer: if Request was not properly initialized do it
*DataManagement
FIX: Remove use of ReplicaManager, because RepMan is deprecated
[v22r0p6]
*Workflow
FIX: Solve problems from selecting cvmfs code and not setRequired parts
FIX: SLIC Module not finding installation directory
FIX: stdhepcut: outputfile was not found
[v22r0p5]
*Workflow
NEW: Added tests, improved logging output
FIX: UploadOutputData
FIX: FailoverTransfer
FIX: UploadLogFile
FIX: ModuleBase
[v22r0p4]
Workflow
*FIX: Fix OutputDataPolicy
[v22r0p3]
Workflow
*FIX: Restore OutputDataPolicy
[v22r0p2]
*Core
NEW: Based on Dirac v6r11p17: DO NOT USE
FIX: Some bug fixes, now jobs are running locally
[v22r0p1]
*Core
NEW: Based on Dirac v6r11p17: DO NOT USE
FIX: InputData Executor not working
[v22r0p0]
*Core
NEW: Based on Dirac v6r11p14: DO NOT USE
[v21r5p2]
*Core
FIX: dirac-ilc-add-user script
[v21r5p1]
*Core
NEW: Same as for v21r0p0, but using Dirac v6r10 instead of v6r9
[v21r0p0]
*Workflow:
FIX: Kept changes to RCSID and HeadURL and whitespace for pylint
FIX: tried to exclude everything related to (Set)Required
*Interfaces
CHANGE: LCApplication class, between Application and Applications. Allows the definition of the steering files per application
CHANGE: LCUtilityApplication: Needed for the steeringfiles handling Refactoring of Application classes
*Scripts
FIX: dirac-ilc-show-software: If an application has additional sections (AdditionalEnvVar) but no Dependencies dirac-ilc-show-software would crash.
FIX: dirac-ilc-add-whizard: Fix passing full path to the tar call instead of the relative path
FIX: dirac-ilc-add-software: cleanup
*Core
NEW: Include functionality for CVMFS (S.Poss)
*Workflow
NEW: StdHepCut: Allowing to specify cuts inline (S.Poss)
NEW: HandleProdOutputData removed, was not used
*ILCTransformationSystem
FIX: Cleanup of DataRecoveryAgent
[v20r1p10]
*Core
FIX: small problem in fullCopy
*Workflow
CHANGE: ModuleBase: leave the fullCopy decide if the file is here of not.
[v20r1p9]
*Core
NEW: fullCopy now accepts \*items. Will not copy all items from the base directory, nor all items from all
sub directories
*Workflow
CHANGE: ModuleBase: Use new fullCopy signature.
[v20r1p8]
*Workflow:
FIX: ModuleBase: in case there are no required items.
[v20r1p7]
*Core
FIX: the fullCopy utility needs more work.
*Scripts
FIX: dirac-ilc-show-software: If an application has additional sections (AdditionalEnvVar) but no Dependencies dirac-ilc-show-software would crash.
[v20r1p6]
*Worfklow
FIX: MarlinAnalysis: Marlin dll in case of user libs
[v20r1p5]
*Workflow
FIX: MokkaAnalysis: copy the mac file to the run directory
[v20r1p4]
*Workflow
FIX: Request name
[v20r1p3]
*Workflow
FIX: cat is not echo
[v20r1p2]
*Core
NEW: fullCopy utility (first try)
*Interfaces
NEW: setRequired method for the applications
*Workflow
NEW: Handle the Required
FIX: adapt more to the Failover client
[v20r1p1]
*Workflow
FIX: signature of failover methods
[v20r1p0]
*all
CHANGE: merge CVMFS branch with trunk, implement the changes in v20r0pX
*ILCTransformationSystem
NEW: TarTheLogs agent that takes care of tarring and putting on castor production logs for stopped prods
[v20r0p7]
*Core
FIX: bad tag name
[v20r0p6]
*Core
FIX: :any:`checkCVMFS` method
*Interfaces
FIX: LCApplication: only add the steering files to the soft to install if it's been defined