eromancer
patreon
Inner Circle Update for May 16!
🕑 Added 2018-05-17 02:22:20 +0000 UTCThis is a big one guys!
Following on the heels of what was a really good first week for May, we've got some big news, and also some major craziness to deal with! Ultimately it's good news, but it means a lot of unexpected work for us in the short term. If you remember back in August of last year we did a study to see how viable it would be to ditch Daz and move everything character-related to Maya. The plan was to look at it again after v0.06, but newfound rendering issues with Daz's Iray engine require us to make this jump now. The nail in the coffin was that early this week we discovered none of our workflows for creating liquid are reliable because Daz has serious issues with how it renders refraction and subsurface scattering. The good news is that converting our characters to Maya is now far easier than it was in August. Not only has Daz added export mechanics that help with the process, but there's now a third-party tool which, while it doesn't solve all our problems, knocks out a lot of the heavy lifting work. At any rate, I'll leave the gory tech details for below -- hopefully after you see the menagerie of Daz rendering errors you'll come to the same conclusion as us :D. Effectively though, this means we need to put our to-do list for this month on hold and address this immediately, then pick up where we left off once we are up and running with character stuff in Maya.
So, now for the big news! AltairPL has been tossing this around for a bit and is now at the point where he's decided he wants to make a major effort to attempt having the new URGE engine ready for v0.06. If it works out, this means RPG Maker will (on your guys’ end at least) be left in the dust. Kind of fitting that we're shedding all our noobie skins simultaneously, really.
Anyways, aside from the details on the above, we have some other stuff to show off as well :D. Check it all out below!
Onyx’s New Top + Props by Mr. Kittyhawk
Let’s start light! Our newest artist, Mr. Kittyhawk, has finished his introductory task! He remodeled and retextured Onyx’s crop top shirt from scratch, and the difference is night and day. The material portion turned out to be quite difficult, as this silky look we were after isn’t something any of us had done. Uber and I jumped in and got to experiment a lot to figure out how to achieve such a look. Here are some images!
In addition, Mr. Kittyhawk is now working on props for the Void’s marketplace. He’s starting out with a few crazy display apparatuses similar to ones seen throughout cyberpunk media. The one he’s working on now is providing a reason to learn some physics simulation for wire placement. Here are some screens of that:
- Display Apparatus 1
- Display Apparatus 2 (wire simulation in progress)
Fluids Progress
Before all the rendering issues came to light this week, we made some great progress following up with creating geometry from liquid textures. Uber’s Houdini networks are now capable of pretty much any operation we could want for manipulating the format of and prepping liquid geometry (though some work still needs to be done to make it quick to use). We can:
- Take textures of liquid painted on the characters and turn it into geometry (as discussed last week)
- Turn the geometry into voxels, making it an actual volume instead of polygons (this is useful for further processing). Excuse the use of this crappy test blob :D.
- Perform Boolean operations. This means we can merge multiple pieces of liquid geometry to prevent seams, and we can also cut liquid geometry that’s clipping away.
Unfortunately, even with all of this, there are zero options for skirting around Daz/Iray’s rendering issues with liquid. We’re getting great results in Maya/Redshift however! Which leads us to…
Why We’re Converting to Maya Now
I’ll try to make this as painless as possible :D.
In the event you weren’t around when we looked at doing this previously, the idea is that we will convert our Daz characters to Maya and do all posing/animating/rendering/whatever there and only use Daz for what it’s good at: piecing together character concepts and shaping bodies. Since we already do all our environments in Maya, there are a lot of long-term benefits to having everything under the same roof. There are also a lot of restrictions lifted for us by using Maya over Daz, some of which I’ll list further down.
First though, here’s what inspired us to jump ship on Daz/Iray now instead of after v0.06, as well as proof that everything works in Maya/Redshift:
- The Iray rendering engine simply cannot handle opacity and volumetric physics simultaneously. I actually reverse engineered the MDL shader code to insure it’s the rendering engine and not just the shader. This means if we add a texture layer of liquid over a character, we can’t mask off the area where we want the liquid with an opacity map. If we do, the liquid uses what’s called “thin-walled refraction”. It looks like a bubble instead of a solid. This means shadows won’t render inside the liquid, and on skin clear liquid becomes nearly invisible. The reason we didn’t catch this sooner is that our liquid simulation routine doesn’t use textures – it uses solid geometry – meaning we don’t need the opacity map. Malise’s suit is black, so testing on that was a bad idea. Lastly, the issue is not quite as obvious on cloudy liquids like slime and cum. On the flip-side, here is a test I did showing this works perfectly in Maya/Redshift.
- Daz/Iray glitches out when a solid-body refractive object clips with other geometry. It renders shadows incorrectly because it can’t see the back side of the refractive object, meaning light doesn’t refract properly. Here’s a water drop test demonstrating the issue, and here’s the nastiness it can create with solid liquid geometry clipping on a character’s skin. To boot, you can see a secondary error in that picture, where the geometry underneath the skin causes errors in the subsurface scattering of the skin (the yellow). We developed the Houdini Boolean operations to remove clipping liquid, but it looks equally bad in most situations because the liquid looks more like a glass shell. But, here is the water drop test working perfectly in Maya/Redshift.
- Subsurface scattering doesn’t work at all on non-volumetric geometry in Daz/Iray. If added to a plane, the edges of the plane will bleed light and become infinitely bright over time. This means cum and ooze *must* use 3D geometry in Daz/Iray. Here’s what happens if we try it with a displacement texture. Doing it with geometry would be fine, if not for the above issue where 3D geometry can’t clip with the character’s skin/clothes. Basically, it turns into a catch 22 of unsolvable rendering errors.
Here are a few more images showing things working great in Maya/Redshift.
While the above was the nail in the coffin for Daz/Iray, a push off the ledge is the fact that it will be a lot easier to convert our characters to Maya than it was six months ago. So, here’s a list of some of the really good benefits we will gain from this switch in the long run (some of these are really technical):
- Our environments and characters will use the same rendering engine / shaders, meaning we can render characters within scenes instead of compositing them in post. This should be a big benefit for cutting down time spent on cutscenes.
- Automated rendering of poses. Maya has a keyframing timeline that isn’t completely riddled with bugs, allowing me to keyframe poses instead of keep them in separate projects. This would save me hundreds of hours over the course of the project, especially in the event of a full re-render.
- Having access to physics simulation without having to export/import between Daz/Maya will be a nice benefit.
- Compatible character rigs for literally any other popular 3D app besides Daz. There’s even a bridge to Unity/Unreal (hello future projects?).
- Physics simulation for posing said rigs -- something not possible previously since we couldn't bring character rigs to Maya.
- We can make custom rigs for non-human enemies in Maya.
- Animation does not, from what I understand, suck in Maya. I have to learn it, but I can say with certainty that it’s far superior to Daz’s. We could even progress to motion capture at some point.
- We can freely use displacement and high poly models in Redshift. Remember how we had to remodel Neon and Ven’s armor from scratch because Iray bugs out and reverts to CPU rendering when heavy displacement is used?
- We will have access to render layers in Maya.
- Maya has Hypershade, a node-based tool for modifying, creating, and feeding input to shaders. Daz doesn’t expose things at this level, meaning there’s way more texture editing.
- Wayyy more flexibility with rendering. You can selectively turn shadows on/off in Redshift. Not being able to with Iray has previously complicated our workflow and would inevitably cause difficulties in rendering cutscenes. You can hide objects in Redshift, but still generate light/shadows from them. You can put lights directly behind geometry utilizing opacity maps (like hair), and not get crazy artifacts (Iray would create a bright white ring around the hair). The list goes on.
- Redshift has bucket rendering and instant access to AOVs within the viewport.
- Daz’s implementation of Iray only allows for brute force rendering for global illumination – and it can’t be turned off. Redshift can turn off global illumination and has the must faster irradiance cache and irradiance point cloud global illumination engines. Basically, we should have way faster character renders.
Of course, there are some trade-offs to this, with most of it being in the form of additional work we have to do to get back to where we were. That’s what we’ll look at in the next sections.
Maya Conversion Progress: Materials and Rendering
Long story short: we need to recreate our Iray materials for the existing characters in Redshift. Skin, hair, the whole shebang. That’s not as scary as it sounds though. Since they are both physically based rendering engines it’s largely a job of acclimating to the differences between all the knobs and gizmos and doing the grunt work :D. I’ve actually already accomplished a lot in the last few days with Ubercharge’s help, despite the fact that I’m overcoming the serious learning curve of adapting to Maya’s UI. Ultimately, redoing the materials in Redshift should yield an upgrade as we have a lot more flexibility with Maya’s Hypershade node network. Currently the most difficult thing is skin. Redshift’s subscattering model is different than Iray’s, and will probably take me a couple more days of testing to optimize the results. Here’s my current best attempt (Redshift on left, Iray on right). That said, being able to use Hypershade has already allowed me to get way more detailed for close-ups.
Here are some other screens showing my progress:
- Malise’s face skin details from early yesterday (further along now)
- Body detail 1
- Body detail 2
- Body detail 3
- Eyebrows/eyelashes featuring Redshift’s hair shader
Anyways, this will be my main priority for the near future while TK works on rigging stuff.
Maya Conversion Progress: Rigging
TK here!
So, we decided to transition fully to Maya, and just use Daz Studio for creating the characters. This is quite the challenge for me! Character rigging is typically a specialized job, and I'm not too skilled in this area. Lots of heavy math, programming and scripting go into creating a good rig. Having a good set of controls on top of a skeleton is super important for animating and posing 3D characters. There are a number of available scripts out there to help here, but I had two major criteria. First, the initial skeleton and skin weighting must be kept intact or have a way to come back to the start once the control rig is created. Second, the initial geometry needs to be untouched so we can continue to add morphs to bodies and clothes coming from other Maya scenes or out of Daz. Many of the available scripts will botch #1 in one way or another; they're mostly designed for people with little to no experience and assume that the user doesn't have a skeleton bound first.
Eromancer had previously found a "Daz to Maya" script that I thought was mostly useless for our needs; it automates setting up a character for Maya's HumanIK middleware. Turns out it's not that bad, good even! It's not as fully featured or customizable as other rigs out there, but it takes a huge weight off of me. After some poking around I found that it meets my two basic criteria without problem. It's also a step up in comparison to using Daz for posing.
HumanIK doesn't handle faces or come with preset controls for some of the basics, however. It's handy to have a single slider that poses a hand into a fist or sets a facial expression for example. And that's what I'm working on at this moment. In the end, I'll have a custom script that can import a character of any shape and size from Daz and do all the setup through HumanIK, face control, additional rig controls, and whatever other small hurdles I might come across from here to there.
AltairPL’s Coding Progress and URGE Engine Update
Last IC update was in the middle of my work on fixing various issues with holds/multi-teaming. All the major ones were eradicated, and the only two related things left to do is fixing a few smaller issues and testing the hell out of it. I could and probably should tackle them ASAP, but I really had enough of watching and coding battle stuff, so I decided to move to something else for the time being.
When I looked at the TODO list in monthly update I've noticed two map lighting entries with my name on it. After we (internally) discussed map lighting last time, I started thinking about something. A problem with map lighting is that it requires usage of some color blend modes not available in RPG Maker. I did a prototype implementing them in Ruby, but there are two major issues with it that we talked about in a previous update:
- is unbelievably slow - like 60 FPS max and added frame skip once in a while.
- requires completely different processing than URGE, which adds whole new level of complexity to an already complex system
The second issue is the most problematic for me, since I don't like to waste time implementing something complex that will be used only temporarily, so I started thinking about pushing the URGE engine forward. Now that I got reminded about it, I decided to do just that. I’m not sure if I'll be able to fix every issue before 0.06, but I'll do my best to implement and fix at least the most important things. And there is a lot to do to achieve this - here's the list of things to do to make URGE production-ready:
- [REQUIRED] code shader version of Viewport's color, flash, and tone handling;
- [REQUIRED] finish implementing Window class - too much convoluted work to convert used vanilla windows to my own Window ruby implementation
- [REQUIRED] figure out directory contents handling and use it to recode font processing, including added handling for system fonts, and design/implement creation of encrypted archive(s)
- [REQUIRED] fix whatever critical errors we encounter
- [OPTIONAL] fix font issues, especially italics one
- [OPTIONAL] test and, if possible, implement game controllers support
- [OPTIONAL] finish implementing scene transitions - for now Viewport objects are used for this purpose even in RPG Maker, so it can wait
Anyway, I started with updating some libraries, which of course resulted in some unexplained crashes and weird behavior, so I had to figure out what's wrong and how to fix it. The good news is that new versions fixed some problems I was having, so I can remove them from my notes and get rid of some ugly workarounds already implemented. The bad news is that some old issues were joined by some new ones and I will need to address them sooner or later.
The next thing on my list was to change the toolchain used when building the engine application. One of the reasons for this is the probability of making 64-bit version of the engine in the future, which wasn't supported in the old toolchain. Another batch of fixes related to the change was required, but font processing which was broken in the process is still awaiting recoding. The only bright side of this is that it had serious problems with unicode paths, so I would need to fix it sooner than later anyway.
Ever since I started working on the URGE, I made a lot of basic mistakes related to development environment, and it seemed like a good time to alleviate this now, so I've finally properly organized development environment and in the process, I've also revamped configuration file used when compiling the executable. This took me a long time but was well worth it. Not only it's easier to edit and maintain but should also allow building 64-bit version of the engine. Unfortunately, I won't be able to test it for some time, since I don't have time to compile 64-bit versions of some required libraries.
With this out of the way, there was only one last thing to do before returning to URGE proper: updating and fixing the loader executable. Suffice to say, it's finally fully compatible with unicode paths and 32/64-bit detection seems to be working just fine, so it's another step towards possibility to have 64-bit version of the engine.
While doing all this, I also started making my own private library containing a lot of useful functions and macros, which should help in keeping the main application code cleaner and clearer. Can't say I finished it, since I will be updating it as I go, but it already contains few functions which will be used a lot.
Finally, after so many derails, I got back on track and started coding something that will be needed by the engine. I started with making a prototype function outputting list of the directory contents to the console, which will be later used as a base for many other functions. I've picked this one from the TODO list for two reasons:
- Two features necessary to use URGE in production require processing of all files in a directory.
- One of those two is font processing, without which I can't run and test MATM stuff in URGE.
Anyway, directory listing sounds pretty easy, but it's in fact very convoluted at the level of C programming language. At the moment I have a prototype of the directory listing function, which I'm still working on. Works for the most part, but there are still some problems with it, especially with recursion of unicode paths. In case you haven't noticed, unicode paths are my bane, but I think I started getting the hang of that, so I should be able to fix the function and implement additional functionality to it soon(ish).
That's about it for this update. My immediate URGE TODO list looks like this, if anyone's interested:
- Finish directory listing function which will be used as a base to at least 2 other functions needed sooner rather than later;
- implement some things I've learned while doing the directory listing function to other parts of the code;
- improve error handling - more error checks and more information provided to the users (so they can pass it to us ^^);
- go through the code and replace workarounds to old issues with proper code;
- go through the URGE notes gathering dust and see if anything there is worth implementing at this moment.
Comments
Jamie C.
I had my doubts for a while but after the transition I was convinced that I was still in this til the end. hoping there will be a sequel to this game. :D
Her shirt was the worst thing? Nothing a little under-boob didn't fix.
MangoFishSocks
A running joke? That's way too harsh. Especially with how transparent and communicative they've been in and of itself and compared to other adult game devs. Almost guaranteed that comes from frustration, but I think the situation is the most frustrating for the team by far, given how little appreciation their hard work is being given on various big platforms. I hope they'll just move along and ignore the pessimists who see all of these updates as "a scam."
dd209
Have to say the same thing, frankly. I've defended you guys for a while but over the past year I can't remember how many times I've heard this song-and-dance about restarting something or other and "this time it will totally speed up the project in the long term, honest guys!". I've dropped almost 300 dollars on you guys since around 2014, and whilst there was a good period where you were pumping out new content pretty regularly, you guys are now a running joke on ULMF, 4chan and f95zone. I'm sorry if you guys are doing your best, but at the end of the day actions speak louder than words, and the state of the project over the past year really says it all. Unfortunately I'll be deleting my pledge and will see how things are going in maybe six months or so.
Thanks :D. We're not veteran game devs at all -- even our heavy hitter has never worked on a game. I realize it can be frustrating watching us make errors and have to double back on stuff, but if there were a ton of pros making adult games we wouldn't need to. I therefore try to be as transparent as possible. For people that get it, cool, but more importantly it acts as undeniable proof that we're putting a lot of effort it even if things aren't successful and we have to change course later. We haven't updated the rest of Onyx yet -- just her shirt since it was by far the worst thing heh. She'll get some serious improvements once we bring her over to Maya though (which is going super well by the way!)
as i can tell as a "not so long time follower" on the project: eromancer and his team did a nice work so far. even that i would wish to get more frequently updates on the project he always explained everything well and detailed with a nice view (picture as well) on the progress with an detailed explaination of all the tiny devils inside the problems. So far! i'm not complaining about subscribing to this project. It's quite worth it so far. the quality went from average to over the top in my eye. (Little side note: new onyx render when? =P )
Jon P
Very informative comments on this post, I love 'em. Ero - I'm glad you're committed! It really does show in the tech update posts that you and your team aren't wasting time. Instead you made a hard call to redevelop almost from scratch in an attempt to get away from RPG maker and make AAA+ studio quality art. I'm convinced it was the right move, and I don't regret backing you the last 2 years even one single smidgen. That said, for your wallet's sake I hope URGE and H content are ready soon!
I actually brought that up the other day Antilles =D. The idea was always to switch to v0.10 or something when we switch to URGE
Somnovox
Very cool. ^_^
or even matm 2 :D
MangoFishSocks
Thanks, that makes a lot of sense.
Antilles
With the art overhaul, the switch to Maya, and possibly URGE as well, I feel like the next version deserves to be v0.1.0 at this point.
Same principle Mango, but when referring to polygons the name 'Boolean' is referring to the logical operations performed on them. The diagram on the right side of this wikipedia page is a perfect example of what we're doing <a href="https://en.wikipedia.org/wiki/Boolean_operations_on_polygons" rel="nofollow noopener" target="_blank">https://en.wikipedia.org/wiki/Boolean_operations_on_polygons</a> @ogre five, true maybe, but a lot of people here want their H content in gameplay format. Once we're back to putting out regular content we should be golden. I'm obviously not expecting pledges to come flooding in while we are basically giving tech dissertations on H game development -- I'm simply trying to weigh development priorities vs available funding.
No worries, and thanks for the support :D. I didn't mean for the "I'm losing money" comment to sound so "woe is me" -- I was merely making the point that I'm making a financial investment in this as well. I've never intended to make a bunch of money off the development process. People will expect big things out of v0.06 when it's finally done, and I'm really optimistic that with the direction we're heading it'll bring in enough funding to maintain our current team and even expand it further.
ogre five
I think if you guys dropped some H-content each month it would help. Just a render or two of one of the characters doing something X rated, or a pin up pose. Wouldn't need to be perfect, just something
Essentialy what we do here is not only simple update v0.06 but almost like building new game from scratch new engine new tech for maps for characters for compositing new UI I would say this is our last and most important big tech overhaul and was the very best long term choice we could do, also every small tiny tech decision will multiply exponentionaly in future
.06 is an arbitrary number and given the work and progress over the past year and having worked in software dev for 20 I would say it is off the mark. I am sorry to hear that you are losing money on the project as you really are doing good work. More than once you mentioned releasing another beta and something has come up or enhancements were prioritized. That is the main source of frustration and despite the wonderful technical posts you make it rings hollow after a while. If anything is going to reverse that trend of losing money it will be delivering something. When that day comes I will sign back up. I do hope you make money on this project as the work is excellent and wish you the best.
MangoFishSocks
While I can understand his/her frustration, I disagree with it. Clearly a lot of progress is being made, especially since the start of May, and you clearly put a lot of care and effort into both showing that progress and showing us that our money doesn't go to waste. It also feels like you're reaching the light at the end of the technical development tunnel with model, light and fluid networks being rounded up, even with this Maya setback, which means all of the progress and pledges will bear fruit. If I'm wrong on that, do tell. You've also done a great job at explaining the more technical stuff to the less tech-knowledgeable people. Unlike an earlier, fairly recent inner circle update, I actually understood most of it. Mr. Kittyhawk's material on the crop top looks great. Proper soft fabric. Hopefully it'll interact well with fluid, soaking in and whatnot. Also question about the fluids. You say that Uber's Houdini networks are capable of performing Boolean operations to mix different bodies of fluids without seams and cut liquid away. Since a Boolean is a 0 or 1 variable, do you literally just select a piece of liquid and 0 it away? Or are you talking about a different kind of Boolean I've never heard of in the basic Game Development course I did in school?
@FraggleRump (why doesn't the Patreon mobile app have a reply function :( ) while it's understandably frustrating, remember that 0.06 is an arbitrary number. Also consider Ubercharge was part of an Oscar winning VFX team and doesn't come cheap. I'm losing money on the project and want to get v0.06 out as soon as possible, but I feel this is a coin toss worth taking.
milking and running into complications aren't the same thing.
release date of .06? I have donated over a year straight and honestly the work your doing is very good but its been well over a year since you delivered anything substantial. It feels like you are milking the support that is given you. I advise releasing .06 with current and take your time for the next version. we are talking .06 here. For the record I deleted the pledge. I do think you do good work but the constant moving of the goal post is too much.
