For beta testers : AQE 0.33.0.6 available.....

[SAPSTAR]

What I meant is I got the following generated ECL :

Code:

qmat_idx=-1
qmat_name=BVCD Underground
qmat=
Threads=-1                          <------------------ It shouldn't be just after the "qmat=" line.
16 18 20 22 26 28 32 39
18 20 22 24 28 32 39 44
20 22 26 28 32 39 44 48
22 22 26 32 39 44 48 54
22 26 32 39 44 48 54 64
26 32 39 44 48 54 64 74
32 39 44 48 54 64 74 84
39 44 48 54 64 74 84 94
20 24 26 28 38 42 46 53
24 26 28 38 42 46 53 58
26 28 38 42 46 53 58 62
28 38 42 46 53 58 62 68
38 42 46 53 58 62 68 78
42 46 53 58 62 68 78 88
46 53 58 62 68 78 88 99
53 58 62 68 78 88 99 99

Which is wrong it should be (for example) :

Code:

threads=-1
qmat_idx=-1
qmat_name=BVCD Underground
qmat=
16 18 20 22 26 28 32 39
18 20 22 24 28 32 39 44
20 22 26 28 32 39 44 48
22 22 26 32 39 44 48 54
22 26 32 39 44 48 54 64
26 32 39 44 48 54 64 74
32 39 44 48 54 64 74 84
39 44 48 54 64 74 84 94
20 24 26 28 38 42 46 53
24 26 28 38 42 46 53 58
26 28 38 42 46 53 58 62
28 38 42 46 53 58 62 68
38 42 46 53 58 62 68 78
42 46 53 58 62 68 78 88
46 53 58 62 68 78 88 99
53 58 62 68 78 88 99 99

I'll test your new version and let you know.

[Darksoul71]

OK, I understand ! I´ll fix this with the next release.

TTYL,
D$

[BR7]

@Darksoul71
Do you have any ideas why I was getting errors using qmatop? Also could you tell me the speed increase on the last test I sent(when you have time) it's making me curious 
Keep up the great work

[Darksoul71]

@BR7:
Thnx again for doing all the testing but due to my stupidity the results are useless ! 

I have no clue what was causing the QMatOp error.

Nevertheless here are the encoding times on your Pentium D.
It´s really no surprise that two multithreaded instances of AQE run slower than one

With QMatOp:
AQE encoding time: 21 minutes
AQE^n encoding time: 26 minutes

Without QMatOp
AQE encoding time: 16 minutes
AQE^n Encoding time: 16 minutes

I´ve change my code a bit in order to avoid screwing up a custom matrix when writing the "thread" value to the ECL file.
The change is completed but I wanna do some testing myself prior I release this...

Thanks for all your support,
D$

[SAPSTAR]

@Darksoul71
Do you have any ideas why I was getting errors using qmatop? Also could you tell me the speed increase on the last test I sent(when you have time) it's making me curious 
Keep up the great work

It seems you both missed my previous post. ..saying that I found a bug about that. Quickly, the reason is the generated pictures used by QMatOp are not unique...I changed that in the current version I have( 0.33.0.7 not yet released). And now it's perfectly working. I may release it soon but I would like to be sure no other "bug" is there.

[BR7]

Thanks for the heads up SAPSTAR  I will be looking forward to your next release

[Darksoul71]

@SAPSTAR:
Oh, I´ve not payed attention to this posting.
Sorry....

I´ve just uploaded the latest version of AQE^n and you can grab it here:
http://de.geocities.com/dark_soul_71/AQEN.zip

Based on your comments I´ve modified my INI routine for the ECL file and it should not screw up a custom matrix anymore if the threads value is not part of the ECL file. At least I didn´t see any problems in my tests. Disabling MT works as well as limiting the numbers of threads assigned to AQE. You can define this with the threads value.

To make the operation of AQE^n more "visible" I´m using the RunHidden value in the ini also for DGIndex. So if you´ve choosen to run AQE visible, DGIndex will be visible (but minimized) as well. This should also help you for better verification if something did go wrong or AQE^n simply merges some M2V segments.

Since I moved the value "RunHidden" from the AQE section to the common section you can either copy it over manually or delete your existing ini-file and let AQE^n generate a new one.

BTW: I´ve included the AutoIt source for AQE^n in this zip in case anyone would like to have a look at.

@BR7: I think if´s safe to do tests with AQE^n once SAPSTAR releases the version with the "fixed" QMatOp code. I´m really curious how two single threaded instances of AQE compare to one multithreaded instance on your system.

Later,
D$

[SAPSTAR]

@SAPSTAR:
Oh, I´ve not payed attention to this posting.
Sorry....

I´ve just uploaded the latest version of AQE^n and you can grab it here:
http://de.geocities.com/dark_soul_71/AQEN.zip

Based on your comments I´ve modified my INI routine for the ECL file and it should not screw up a custom matrix anymore if the threads value is not part of the ECL file. At least I didn´t see any problems in my tests. Disabling MT works as well as limiting the numbers of threads assigned to AQE. You can define this with the threads value.

To make the operation of AQE^n more "visible" I´m using the RunHidden value in the ini also for DGIndex. So if you´ve choosen to run AQE visible, DGIndex will be visible (but minimized) as well. This should also help you for better verification if something did go wrong or AQE^n simply merges some M2V segments.

Since I moved the value "RunHidden" from the AQE section to the common section you can either copy it over manually or delete your existing ini-file and let AQE^n generate a new one.

Wonderful!!! I'll start testing it wth several kind of encodes + combinations!! Thank you for your work!!
I have something to check...did you ever notice a slight pause at each segment merge point?! I think I noticed it; I have to check it again, but I'm wondering if it couldn't be the number of frames in your AVS scripts ?! one missing or one additional.....

[Darksoul71]

Hi SAPSTAR,

normally there should be no problem with this.

I´ve just manually checked the start / endframe of two segments and they looked identical to the corresponding frame in the source script. I also checked the length of the merged segments and they have the identical frame# as the source script.

You can easily see what AQE^n does work like from the log file:
02:20:29 >>Width:     720
02:20:29 >>Height:    576
02:20:29 >>Frames:    6535
02:20:29 >>Framerate: 25,000
02:20:29 >>Number of Cores: 2
02:20:29 >>Calculated Range# 0: 0-3268
02:20:29 >>Calculated Range# 1: 3269-6535

The only thing I could imagine that DGIndex causes a "hickup" in the merged M2V stream but I can hardly imagine that. DGIndex and DVD2AVI are used all over the place and no-one reported "hickups" in the resulting demuxed stream. I do some further investigation on some merged videos this evening.

Just a short report from Revgen. He did some test on his X2 4800+ and here are the results:
AQE Automated MultiThreading = 269seconds
AQE MultiThreading OFF = 347 seconds
AQE^N with Multithreading OFF in ECL file = 353 seconds
AQE^N with Multithread On in ECL file = 353 seconds

This is really strange and I have to followup with Revgen if AQE was really running with singlethreaded mode.

Very interesting topic and the worst thing I could imagine it that AQE^n is useless or only provides a marginal speedup on Pentium D CPUs. We will see

Best regards,
D$

[SAPSTAR]

Hi SAPSTAR,

normally there should be no problem with this.

I´ve just manually checked the start / endframe of two segments and they looked identical to the corresponding frame in the source script. I also checked the length of the merged segments and they have the identical frame# as the source script.

You can easily see what AQE^n does work like from the log file:
02:20:29 >>Width:     720
02:20:29 >>Height:    576
02:20:29 >>Frames:    6535
02:20:29 >>Framerate: 25,000
02:20:29 >>Number of Cores: 2
02:20:29 >>Calculated Range# 0: 0-3268
02:20:29 >>Calculated Range# 1: 3269-6535

The only thing I could imagine that DGIndex causes a "hickup" in the merged M2V stream but I can hardly imagine that. DGIndex and DVD2AVI are used all over the place and no-one reported "hickups" in the resulting demuxed stream. I do some further investigation on some merged videos this evening.

Just a short report from Revgen. He did some test on his X2 4800+ and here are the results:
AQE Automated MultiThreading = 269seconds
AQE MultiThreading OFF = 347 seconds
AQE^N with Multithreading OFF in ECL file = 353 seconds
AQE^N with Multithread On in ECL file = 353 seconds

This is really strange and I have to followup with Revgen if AQE was really running with singlethreaded mode.

Very interesting topic and the worst thing I could imagine it that AQE^n is useless or only provides a marginal speedup on Pentium D CPUs. We will see

Best regards,
D$

OK for the length of the movies...I have to check again for this hickup.
About the results of Revgen, I got the same kind of result with an episodic animation :
AQE Automated MultiThreading = 421 minutes
AQE^N with Multithread On in ECL file = 450 minutes

I didn't test the other combinations yet for a reason of time....I dont think we should take care of small segments for the measures. On small segments the slightest event could change the final time. I really think we should consider on;y whole DVDs/Movies for the time results. Dont you think?!

[Darksoul71]

AQE Automated MultiThreading = 421 minutes
AQE^N with Multithread On in ECL file = 450 minutes

IMO the standard comparison should always be:
AQE Automated Multi Threading vs. AQE^n without Multi threading

I would expect that the runtime of AQE^n without multithreading is <= AQE with automated multithreading, given that a longer movie is used for testing.

Results of running AQE^n with enabled MT only provides a comparision wether AQE with automated multithreading really "eats up" all your CPU time. The only platforms I can think of where running AQE^n with MT enabled would be either a dual core CPU with hyperthreading technology (P4 Extreme edition) and dual Opteron AMD workstation. With 2x2 (virtual) cores rep. 2x2 "true" cores multiple instances of AQE running multithreaded should really shine from a performance point of view.

I really think we should consider on;y whole DVDs/Movies for the time results. Dont you think?!

I second that. The longer the encoded movie is, the better we will be able to estimate the speed increase (if there is any).
Smaller segment are not bad but not really representative.

Also the source scripts should have not addtional filtering in them (e.g. Deen, Undot, etc). It´s important to find out maximum performance increase which is possible. It should be a no-brainer that the speed gain using QuEnc^n, AQE^n or multiple encoder instances (as in DVD-RB) are depending on the available CPU time. Add CPU intense filters to your AVISynth script and your possible 60% increase will melt down to 0% or even worse cause a speed decrease compared to only running a single encoding instance.

For people owning a dual core system and using source scripts with heavy filtering I would even suggest to stick to AQE with multithreading enabled and using the MT AVISynth plugin of tsp (http://forum.doom9.org/showthread.php?t=94996)

This should provide better results than using AQE^n. At least that´s my theory...

Regards,
D$

[Darksoul71]

OK, here is an update:
Revgen has prodided new values. As I suspected he didn´t disable MT when using AQE^n.

These are the values he measured with this stopwatch:
1) 441 seconds with AQE Single Threaded
2) 340 seconds with AQE Multithreaded - 23% faster than single thread
3) 308 seconds with AQE^n - 30% faster than single thread

Those are not the pure encoding time but with merging M2V segment.

Here are the pure encoding times from the log files:

1) AQE Single Threaded = 35.000FPS

2) AQE Multithreaded = 47.000FPS

About 65% faster than singlethreaded.

3)
AQE^n001 = 31.00FPS
AQE^n002 = 30.00FPS

Total AQE^n= 61.00FPS

About 74% faster than Single-Threaded

This indicates that the AQE^n routes is still 10% faster but also that the multithreading implementation is also very good if we keep in mind that some encoders for $$$ show only 20-30% increase on dual core / CPU machines.

I´m curious how AQE^n "competes" with AQE with automated MT when QMatOp is enabled.

Also I´m curious what increases BR7´s Pentium D shows when using AQE^n vs. AQE with MT.

Any comments ?

cu,
D$

[SAPSTAR]

Interesting results....but what was the source?!
I will give you my own results soon...with an episodic animation : AVATAR Disc 4

Just a thing I dont agree with your third statement :

3)
AQE^n001 = 31.00FPS
AQE^n002 = 30.00FPS

Total AQE^n= 61.00FPS

It's wrong you cant just add them. You have to get the total number of Frames and then divide it by the total duration (Both encodes finished + merge of movies (must be 308 seconds in that case)). Your result would be true only if the two encodes had the exact same duration which I doubt.....Given the results I would assess the speed at 52FPS but not 61FPS....

EDIT: An easy way to check it...is just the comparison of the encoding time: 3)/1) = 1.43 => AQE^n 43% faster than AQE ST ; 3)/2) = 1.10 => It confirms the 10% faster assessment.

[Darksoul71]

Interesting results....but what was the source?!

I can not comment on the source since SAPSTAR did the test.

It's wrong you cant just add them. You have to get the total number of Frames and then divide it by the total duration (Both encodes finished + merge of movies (must be 308 seconds in that case)). Your result would be true only if the two encodes had the exact same duration which I doubt.....Given the results I would assess the speed at 52FPS but not 61FPS....

Hm, I haven´t seen the log and didn´t do the calculations myself but do not the a mistake here.

In terms of adding the FPS for each AQE instance called by AQE^n I think I would do the same.
Both encodes have the same duration since AQE^n splits the movie into two identical segments.

OK, 1-2 frames differences may be because of rounding but 1 : 2 is 0.5 wouldn´t you agree ?

Also merging the M2V segments isn´t really a big deal. For standard DVD chapters this usually takes only 10-15 seconds on my A64 3200+

In general I see your point. The best approach to compare AQE^n with AQE MT is simply running a full DVD backup (> 90 mins) and compare the encoding times reported in the DVD-RB log file.

I´m interested in your results on your HT machine. If I wouldn´t be on budget and if AMD wouldn´t have anounced a price drop for it´s dual core CPU I would straight go out there and buy a X2 3800+ 

Cheers,
D$

[SAPSTAR]

Hm, I haven´t seen the log and didn´t do the calculations myself but do not the a mistake here.

In terms of adding the FPS for each AQE instance called by AQE^n I think I would do the same.
Both encodes have the same duration since AQE^n splits the movie into two identical segments.

OK, 1-2 frames differences may be because of rounding but 1 : 2 is 0.5 wouldn´t you agree ?

No no I assure you; same number of frames doesn't mean same duration for the encode, it depends on the complexity of the scenes(predictability,....).  Most of the time you will have one encode finished before the other one.
And as I said, by just looking at the times, you can check that AQE ST is not 74% slower than AQE^n.....
AQE ST : 441 seconds
AQE^n ST : 308 seconds
=> 441/308=1.43 => 43% faster (not 74%) => 35 FPS *1.43=50.05 FPS which is really different from 61FPS !!!