db0
patreon
Cancelling and better calculations
🕑 Added 2022-11-03 00:18:44 +0000 UTCOne new feature set has been added to the horde, and this has to do with making the kudos calculations more accurate and tracking everything correctly
The first update is the Kudos calculation. We realized recently that the kudos awarded for generating larger images did not match the comparative time it took to generate them compared to smaller images.
Some experimentation showed that the difficulty of higher resolutions was exponential. So a 1024x1024 image was 6 times as hard as a 512x512 (not 4x as hard as the resolution difference would imply). Likewise a 2048x2048 resolution was 60 times as hard (instead of 16).
After some help by one mathy member of the discord community, we settled on a new formula for figuring out kudos which follows this investigation. On top of that we adjusted the kudos gained by the k_heun and k_dps samplers because they in fact double the steps needed for each iteration, but this was not represented.
The next change came from a request from one of the Sygil-Dev devs, who wanted the capability to generate 4K or 8K images with a comparative amount of steps for it. However as this feature has the potential to choke the horde when a random anonymous account might request it, so I needed a way to ensure this is not abused.
The solution I settled to was that the kudos for extraordinarily large requests (i.e. over 1024x1024 and/or 100 steps) would need the kudos upfront. This would exclude all anonymous accounts since they never have any kudos but also all pure leechers. The people who accumulated a lot of kudos on the other hand would have the option to go wild on the largest resolutions (assuming there's people with beastly machines who are serving them). A maxed out request at 3072x3072x500 would cost a hefty 25K kudos!
The final change in this update was the ability to timeout and resume jobs whose worker couldn't complete them for any reason. Now after some minutes (depending on the resolution), the job will fault and the horde will put it back in waiting mode.
This will prevent a very annoying effect of a worker crashing and causing the whole request to hang until it's cancelled or it goes stale. Now you lose a bit of time but at least you can result assured the horde will eventually resume it.
A few other things were tied to this as well. Now I can check for example how many jobs a worker is dropping due to speed and hopefully get them to adjust their max_power rating to be able to complete at a reasonable speed. It can also help us identify potentially malicious workers who might just be taking requests to drop them
Finally the request cancelling now also correctly burns kudos and rewards workers who've already picked up the job. This is to reward the job already done and to prevent abuse via malicious cancellation.
One last thing is that the horde-wide anti-CP filter has now been significantly expanded. Unfortunately as it is with such things, there is a chance it can lead to false positives. To avoid falling afoul of it, avoid using terms about children with terms about sex or nudity in the same prompt. As always if you're a worker, let me know if anything slips through.
