eromancer
patreon
June Progress Update!
🕑 Added 2018-06-02 11:40:36 +0000 UTCHey guys!
This is going to be a very long update @_@. I have to catch the $3 tier crowd up on a whole lot of action from this month, so I’ll drop the two big bombs right away!
Bomb #1: The image above may not look that special, but is very important because it shows our first full test of a converted, rigged character working in Maya :D. It’s not hard to find errors with it, as we still have a lot of the kinks to work out, but this is the culmination of a very busy past few weeks for us. What does this all mean? It means that we’re following through on our previous effort to get away from Daz and to unify our environment and character workflows in Maya. Why are we working on this now? It was certainly a gamble and a tough decision, but so far it looks like we will be successful, and in doing so will overcome a lot of difficulties we’ve had with Daz and Iray (its rendering engine). Check out the “Why We’re Converting to Maya Now” section of this update for the details!
Bomb #2: AltairPL is making a very strong push to have our new URGE engine ready for the v0.06 release. Assuming this pans out, players will no longer need RPG Maker at all to play MATM, and v0.06 will probably become something like v0.10 to mark this milestone. So far, he’s doing very well, and he has a huge progress report for you.
Bonus! Bomb #3: TK turned 40 the other day. Do your part and chastise him for being an old man :D.
While most of our update will be devoted to these topics, we have a bunch of new props and graphical features to show off, as well as a lot of progress on fluids that we accomplished at the beginning of the month. Check out the contents for this update below! The topics with the *** symbol are new since the last Inner Circle update.
Topics for this Update
- Substance Painter and Fluids
- TK and Ubercharge’s Houdini Magic
- Why We’re Converting to Maya Now
- Onyx’s New Top
- The Void – Market Area Props ***
- Motion Blur
- Armor Damage 3D Mock-Up
- AltairPL’s Coding Progress from Early May
- AltairPL’s URGE Engine Progress Part 1
- AltairPL’s URGE Engine Progress Part 2 ***
- Malise Material Conversion to Redshift
- TK’s Maya Rigging Progress (Week 1)
- TK’s Maya Rigging Progress (Week 2)
- Maya Conversion Progress / TODO List ***
Substance Painter and Fluids
The first week of May was a very good week. We solved a lot of problems that should pave the way for the new liquids workflow, and ultimately will make the process faster and more flexible. Not only that, but we’re now able to convert texture-based liquids to 3D! Soon we’ll have both 3D armor damage and fluids! This is not only a major milestone for being able to use the new lighting system on all artwork; it’s something I’ve wanted to accomplish since starting the project. It’s a big step for us in making quality real-time 3D H-games a possibility down the line.
Last month I talked about using texture-based liquids and starting to learn Substance Painter to be able to paint the textures directly on the 3D models. Uber and TK both have experience with the program, so it’s been a bigger learning curve for me. That said, I have a solid foundation now for the workflow that I’ll likely be using for adding fluids to anything not requiring all-out simulation. Here are some examples of my results for Malise’s high-lust wetness in Substance Painter (keep in mind these are screen caps from within Painter and not renders, so it will look more like actual liquid and not part of the suit when complete)!
TK and Ubercharge’s Houdini Magic
The above was made possible because Uber and TK put a lot of work into a couple of Houdini networks that are able to take image stamps (used for Photoshop brushes) and generate high quality height data, and subsequently create 3D models of the liquid from it. I’ve got a pretty extensive Photoshop brush library devoted just to fluids from working with the old art style, so we’ve been spending time running these through Uber’s converter. Luckily he was able to automate it, so the process goes very smoothly!
Once we have the height data, I’m able to create very accurate Substance Painter brushes that I can use as a starting point for painting on any 3D model (characters/props/whatever). Here’s a comparison showing what I was able to originally achieve with just an image stamp, and what we’re able to do now.
The biggest breakthrough though is that Uber was able to achieve displacement utilizing 3D normals of a model. What this means is I can paint in Substance using these high quality brushes, then send the finished characters *back* to the Houdini network and generate 3D models of the fluids as if they were on the characters (simple test example shown :D). With this kind of capability, we could even rig fluid to follow posing/animations.
We had intended to use the ability to generate 3D fluid models as a much faster way of creating the liquids we need for H scenes within Daz. This is, however where things got sticky (heh), and where we made the call that Daz and Iray were doing a lot more to hinder us than help us.
Why We’re Converting to Maya Now
I’ll try to make this as painless as possible :D.
Following on the heels of what was a really good first week for May, we decided it was time to kick Daz to the curb. Ultimately, it's good news, but it means a lot of unexpected work for us in the short term. If you remember back in August of last year we did a study to see how viable it would be to ditch Daz and move everything character-related to Maya. The plan was to look at it again after v0.06, but newfound rendering issues with Daz's Iray engine require us to make this jump now. The nail in the coffin was that early in the month we discovered none of our workflows for creating liquid are reliable because Daz has serious issues with how it renders refraction and subsurface scattering. The good news is that converting our characters to Maya is now far easier than it was in August, and as you can see from the cover image of this post we’re well on our way to making that happen. Not only has Daz added export mechanics that help with the process, but there's now a third-party tool which, while it doesn't solve all our problems, knocks out a lot of the heavy lifting work. A bunch of tech details follow, but hopefully after you see the menagerie of Daz rendering errors you'll come to the same conclusion as us :D. Effectively though, this means we need to put our to-do list from May on hold and finish addressing this immediately, then pick up where we left off once we are up and running with character stuff in Maya.
In the event you weren’t around when we looked at doing this previously, the idea is that we will convert our Daz characters to Maya and do all posing/animating/rendering/whatever there and only use Daz for what it’s good at: piecing together character concepts and shaping bodies. Since we already do all our environments in Maya, there are a lot of long-term benefits to having everything under the same roof. There are also a lot of restrictions lifted for us by using Maya over Daz, some of which I’ll list further down.
First though, here’s what inspired us to jump ship on Daz/Iray now instead of after v0.06, as well as proof that everything works in Maya/Redshift:
- The Iray rendering engine simply cannot handle opacity and volumetric physics simultaneously. I actually reverse engineered the MDL shader code to insure it’s the rendering engine and not just the shader. This means if we add a texture layer of liquid over a character, we can’t mask off the area where we want the liquid with an opacity map. If we do, the liquid uses what’s called “thin-walled refraction”. It looks like a bubble instead of a solid. This means shadows won’t render inside the liquid, and on skin clear liquid becomes nearly invisible. The reason we didn’t catch this sooner is that our liquid simulation routine doesn’t use textures – it uses solid geometry – meaning we don’t need the opacity map. Malise’s suit is black, so testing on that was a bad idea. Lastly, the issue is not quite as obvious on cloudy liquids like slime and cum. On the flip-side, here is a test I did showing this works perfectly in Maya/Redshift.
- Daz/Iray glitches out when a solid-body refractive object clips with other geometry. It renders shadows incorrectly because it can’t see the back side of the refractive object, meaning light doesn’t refract properly. Here’s a water drop test demonstrating the issue, and here’s the nastiness it can create with solid liquid geometry clipping on a character’s skin. To boot, you can see a secondary error in that picture, where the geometry underneath the skin causes errors in the subsurface scattering of the skin (the yellow). We developed the Houdini Boolean operations to remove clipping liquid, but it looks equally bad in most situations because the liquid looks more like a glass shell. But, here is the water drop test working perfectly in Maya/Redshift.
- Subsurface scattering doesn’t work at all on non-volumetric geometry in Daz/Iray. If added to a plane, the edges of the plane will bleed light and become infinitely bright over time. This means cum and ooze *must* use 3D geometry in Daz/Iray. Here’s what happens if we try it with a displacement texture. Doing it with geometry would be fine, if not for the above issue where 3D geometry can’t clip with the character’s skin/clothes. Basically, it turns into a catch 22 of unsolvable rendering errors.
Here are a few more images showing things working great in Maya/Redshift.
And last but not least, here’s a good example from Redshift of 3D liquid using subsurface scattering atop skin with no visual defects.
While the above was the nail in the coffin for Daz/Iray, a push off the ledge is the fact that it will be a lot easier to convert our characters to Maya than it was six months ago. So, here’s a list of some of the really good benefits we will gain from this switch in the long run (some of these are really technical):
- Our environments and characters will use the same rendering engine / shaders, meaning we can render characters within scenes instead of compositing them in post. This should be a big benefit for cutting down time spent on cutscenes.
- Automated rendering of poses. Maya has a keyframing timeline that isn’t completely riddled with bugs, allowing me to keyframe poses instead of keep them in separate projects. This would save me hundreds of hours over the course of the project, especially in the event of a full re-render.
- Having access to physics simulation without having to export/import between Daz/Maya will be a nice benefit.
- Compatible character rigs for literally any other popular 3D app besides Daz. There’s even a bridge to Unity/Unreal (hello future projects?).
- Physics simulation for posing said rigs -- something not possible previously since we couldn't bring character rigs to Maya.
- We can make custom rigs for non-human enemies in Maya.
- Animation does not, from what I understand, suck in Maya. I have to learn it, but I can say with certainty that it’s far superior to Daz’s. We could even progress to motion capture at some point.
- We can freely use displacement and high poly models in Redshift. Remember how we had to remodel Neon and Ven’s armor from scratch because Iray bugs out and reverts to CPU rendering when heavy displacement is used?
- We will have access to render layers in Maya.
- Maya has Hypershade, a node-based tool for modifying, creating, and feeding input to shaders. Daz doesn’t expose things at this level, meaning there’s way more texture editing.
- Wayyy more flexibility with rendering. You can selectively turn shadows on/off in Redshift. Not being able to with Iray has previously complicated our workflow and would inevitably cause difficulties in rendering cutscenes. You can hide objects in Redshift, but still generate light/shadows from them. You can put lights directly behind geometry utilizing opacity maps (like hair), and not get crazy artifacts (Iray would create a bright white ring around the hair). The list goes on.
- Redshift has bucket rendering and instant access to AOVs within the viewport.
- Daz’s implementation of Iray only allows for brute force rendering for global illumination – and it can’t be turned off. Redshift can turn off global illumination and has the much faster irradiance cache and irradiance point cloud global illumination engines. Basically, we should have way faster character renders.
Of course, there are some trade-offs to this, with most of it being in the form of additional work we have to do to get back to where we were. We’ll take a look at all the progress we’ve made up to this point as well as what we have to do in a bit, but first let’s take a look at some of the other progress we’ve made this month, as well as AltairPL’s progress on the URGE engine!
Onyx’s New Top
Now for a break from the heavy stuff :D. Our newest artist, Mr. Kittyhawk, finished his introductory task early this month. He remodeled and retextured Onyx’s crop top shirt from scratch, and the difference is night and day. The material portion turned out to be quite difficult, as this silky look we were after isn’t something any of us had done. Uber and I jumped in and got to experiment a lot to figure out how to achieve such a look. Here are some images!
The Void – Marketplace Area Props ***
In addition to the above, Mr. Kittyhawk is now working on props for the Void’s marketplace. He’s starting out with a few crazy display apparatuses similar to ones seen throughout cyberpunk media. He’s just about finished with the first one, which has provided a reason to learn some physics simulation for wire placement. He’s also gotten a crash course in creating materials for and rendering with Redshift :D. Here are some screens of his progress:
- Display Apparatus 1 (early model)
- Display Apparatus 2 (wire simulation in progress)
- Display Apparatus 3 (current progress)
- Display Apparatus 4 (current progress - Game perspective)
We will be adding graphics to the screens throughout the area -- some of which are story related.
Motion Blur!
Something we lost when changing to the new rendering style is motion blur. Uber was able to get it working in Redshift easily enough, but for whatever reason Daz’s half-baked implementation of Iray simply doesn’t offer it, so we’ve had to come up with something else. Luckily, it turns out we’re now able to add film-quality vector-based motion blur in post!
The new, more physically accurate art style supports this effect a lot better, and it is something I’ll utilize for action frames, enemy attack frames, and H artwork.
The downside is that Iray doesn’t support the vector render pass needed to achieve this in post, so I have to actually paint it. Luckily that’s very simple (it looks something like this), and I’ll even be able to do it from within my automation network.
Until I’m able to implement it there (I’ll need to reorganize some stuff), I cooked up an example with Malise as a proof of concept for you to check out.
*Note* Obviously this section was written before we decided to make the full switch over to Maya and Redshift. Once we get away from Iray we’ll be able to make full use of this effect straight from the renderer!
Armor Damage 3D Mock-up
Amidst all the work on fluids, and before the craziness with Maya, TK made some progress on Malise’s armor damage as well :D. He masked out the damage on the 3D model so that we could test whether my concept from last month was accurate and to do some preliminary pose testing. We’ve identified only a couple potential problem areas, and overall, it’s way better than anticipated. Once we’re past the Maya conversion he’ll likely refine the mask some more, and then use it as a guide to start the new model. This is one of the things that we’ll have a lot more freedom with in Maya.
AltairPL’s Coding Progress from Early May
Feel free to skip to Part 2 of my URGE engine progress if you read the Inner Circle updates since the last monthly updates!
Roughly around the time the May monthly update was posted, I started fixing the knockdown pose during hold issue. It required making some changes to the portraits system, and while implementing those changes I started thinking about some other changes, improvements, optimizations, etc. Well, this was my first major detour, but I finally got my shit together and fixed the issue with the knockdown.
The fix for that issue was another good use for a portrait system improvement I implemented pretty recently, and since it's working even better than I hoped, I started thinking about other uses for it. I made some extensive notes about it, and while it was still fresh on my mind, I decided to implement one of those things since it was brought up recently during one of the Battle Test releases. I'm talking about player characters changing poses rapidly as a result of enemy actions. This should be much better now, though it's still not perfect, and I'm afraid it's as good as it gets.
After that I fixed various other issues with holds/multi-teaming. All the major ones were eradicated, and the only two related things left to do is fixing a few smaller issues and testing the hell out of it. I could and probably should tackle them ASAP, but I really had enough of watching and coding battle stuff, so I decided to move to something else for the time being.
When I looked at the TODO list in the previous monthly update I've noticed two map lighting entries with my name on it. After we (internally) discussed map lighting last time, I started thinking about something. A problem with map lighting is that it requires usage of some color blend modes not available in RPG Maker. I did a prototype implementing them in Ruby, but there are two major issues with it that we talked about in one of the previous updates:
- is unbelievably slow - like 60 FPS max and added frame skip once in a while.
- requires completely different processing than URGE, which adds whole new level of complexity to an already complex system
The second issue is the most problematic for me, since I don't like to waste time implementing something complex that will be used only temporarily, so I started thinking about pushing the URGE engine forward. Now that I got reminded about it, I decided to do just that. I’m not sure if I'll be able to fix every issue before 0.06, but I'll do my best to implement and fix at least the most important things. And there is a lot to do to achieve this - you can find updated TODO list near the end of my part of this post.
AltairPL’s URGE Engine Progress Part 1
Anyway, I started with updating some libraries, which of course resulted in some unexplained crashes and weird behavior, so I had to figure out what's wrong and how to fix it. The good news is that new versions fixed some problems I was having, so I can remove them from my notes and get rid of some ugly workarounds already implemented. The bad news is that some old issues were joined by some new ones and I will need to address them sooner or later.
The next thing on my list was to change the toolchain used when building the engine application. One of the reasons for this is the probability of making 64-bit version of the engine in the future, which wasn't supported in the old toolchain. Another batch of fixes related to the change was required, and font processing was broken in the process. The only bright side of this is that it had serious problems with Unicode paths, so I would need to fix it sooner rather than later anyway.
Ever since I started working on the URGE, I made a lot of basic mistakes related to development environment, and it seemed like a good time to alleviate this now, so I've finally properly organized development environment and in the process, I've also revamped configuration file used when compiling the executable. This took me a long time but was well worth it. Not only it's easier to edit and maintain but should also allow building 64-bit version of the engine. Unfortunately, I won't be able to test it for some time, since I don't have time to compile 64-bit versions of some required libraries.
With this out of the way, there was only one last thing to do before returning to URGE proper: updating and fixing the loader executable. Suffice to say, it's finally fully compatible with Unicode paths and 32/64-bit detection seems to be working just fine, so it's another step towards possibility to have 64-bit version of the engine.
While doing all this, I also started making my own private library containing a lot of useful functions and macros, which should help in keeping the main application code cleaner and clearer. Can't say I finished it, since I will be updating it as I go, but it already contains few functions which will be used a lot.
Finally, after so many derails, I got back on track and started coding something that will be needed by the engine. I started with making a prototype function outputting list of the directory contents to the console, which will be later used as a base for many other functions. I've picked this one from the TODO list for two reasons:
- Two features necessary to use URGE in production require processing of all files in a directory.
- One of those two is font processing, without which I can't run and test MATM stuff in URGE.
Anyway, directory listing sounds pretty easy, but it's in fact very convoluted at the level of C programming language. Had some problems with it, especially with recursion of Unicode paths. In case you haven't noticed, Unicode paths are my bane, but I think I started getting the hang of that. Well, I thought it will look a bit different in the end, but either way, I can report full success. One function approach changed into few functions approach, but it's much more versatile this way. In a nutshell, name of every directory/file found is passed to another function, which uses it according to the needs, like recursing sub-directories, or checking if file has given extension and processing it if it does. And that's exactly what I needed to fix font processing, which was broken when I switched toolchains and which I would have to recode anyway due to the problems with Unicode paths. Testing showed that Unicode paths are processed correctly, BTW, so it's another small victory. With this, I could scratch two entries from my immediate TODO list, and I could start the game in URGE once again, which allowed me to work on and test remaining TODO entries.
At this point, I wanted to overhaul error handling and reporting, but it kinda overwhelmed me, and since I had no good idea how to do main part of it, and because my old code scares the shit out of me, I decided to leave it for later and deal with some stuff that will allow me to clean up the code somewhat and in turn decrease amount of code needed to go through to do this error stuff properly. With that in mind, however, I did mark all the places in the code I could find that could use more error handling, and keeping it in the back of my head resulted in finding few potential and a bit hidden problems along the way, which will require some attention sooner rather than later.
Next thing I tackled was coding shader version of Viewport's color, flash, and tone handling, which not only are used for some screen flashes during the battles, but they're also currently used in custom scene transition code. Had some serious issues with this, and considering I was having very similar problems with something else shortly after updating one of the libraries, I think it's a bug in said library. Somehow managed to get the undesired behavior under control, but this may not be the last we've seen it, which is kinda bad, since I wanted to use some new features from that library, but now I need to hold off in case the problem gets worse and I'm forced to revert to the older version of the library. Anyway, did what I wanted to do and now shaders are used for viewports as well, and since it was the last thing needing a switch to shaders, I could now clean up all of the non-shader code I kept just in case.
Now that I got rid of the old and obsolete code, I could do what I wanted to do for a long time. Right from the start, all custom RM ruby classes implemented in C were made exactly how are they were seen by ruby, which required a lot of conversion for different types used by shaders and 3rd party libraries. So, I've decided to change them to use data types compatible with internal stuff in hopes that it will improve performance. Data structures were recoded, along with every possible use of them, and functions representing ruby methods were changed to return values compatible with RM. As I've said, I've hoped for some performance improvement, but unfortunately I haven't noticed any change at all, which kinda sucks. Well, at least the code is a bit cleaner, which is always a consolation. Also, while doing this, I've noticed that Window class implementation is in better shape than I thought... It's a bit of a mess that needs a lot more work and cleaning up, but at least it works (for the most part).
Still have few required things on my list, but every time I launched MATM in URGE, one of the optional things was bugging me, so I've picked it to be next. Font drawing library we're using for URGE has some serious issues with drawing italicized text when font doesn't have dedicated italic style, and I'm not even sure if dedicated italic style works, but that's a whole different story. Tried to debug and fix the library itself, but even though I think I know exactly what the problem is, my knowledge was not enough to fix those problems in the library code itself. I've managed to fix one of those problems some time ago by implementing custom outline, but the other one was too complicated for me and I had no idea how to fix it. Problem is that library is using italicized glyph bounding box for position, which is incorrect for glyphs like 'T', 'W', 'Y' and causes them to be drawn/rendered shifted to the left. Had some crazy ideas about how I could fix it, but in the end, something much more simpler, relatively speaking, seems to be working just fine for custom italics, so I'm not complaining. Well, it could use some additional tweaking, but that's something that can be done at a pretty much any time.
AltairPL’s URGE Engine Progress Part 2 ***
In the course of doing of the things listed above, I've located potential deathtrap that could crash the game without any warning or message, and since I had an idea about how to trigger it, I decided to deal with it ASAP. After making a test case and confirming that it's indeed a deathtrap, I've moved to fix it. I was at least a bit remotely related to some other stuff I had in my notes and internal TODO list, so I decided to do few things at the time. This actually took me a lot longer that I planned, since I kept finding other potential issues ranging from small to critical, so I had to either fix them immediately or make notes about what could be wrong and how to verify and fix it. Long story short - in the course of this I've fixed few major/critical issues, a lot of smaller ones, streamlined structures used to represent custom Ruby objects in C, and marked every place where error handling should be added or improved. Since I came back to URGE this time, I've learned a lot of very useful things about C syntax and functionality in general, so I implemented them wherever possible as well. Most of them made the code much cleaner and easier to maintain, which is pretty good for a C beginner like me.
After that I tried doing to things that failed miserably:
- tried to make 64-bit build of the URGE - was building without a problem, which was a success, but kept crashing almost at runtime and the possible reason pointed by the debugger made little sense;
- tried to update used Ruby version from 2.3.x to 2.5.x, which kept crashing at runtime with error code suggesting missing dll or something similar;
Lately, I've been seeing a lot of mentions about some new compiler, so I decided to take a short break from coding and see what it is about. Turns out it's not really new, I just haven't seen or heard about it. Basic info showed a lot of promise, so I decided to dig a little deeper and give it a shot. As it turns out, it's not a better choice for us at this time, but it may be in the future. And while reading about it and preparing for its implementation, I've learned a whole lot about compilation in general and about some other extremely useful stuff. It also resulted in another overhaul of my dev environment, which is now as functional as possible, and shed some light on the two issues mentioned just above.
While trying to compile 64-bit build in new environment I got a lot of warnings and errors, and as it turns out it's a known Ruby issue, so it's out of my hands, and until it gets fixed, or I do something extreme, 64-bit build of the URGE won't be possible. Had more luck with second issue, and URGE now uses newest stable Ruby under the hood, which is a good thing, since IIRC it fixed a lot of Unicode problems on Windows.
Still need to do some changes to my project files related to the dev environment change, but if I'm not mistaken, some stuff I really hate should become at least a little bit easier and less problematic.
Rest of the stuff I did since last IC update was relatively small, but still significant:
- did some research about font rendering, which resulted in finding potential replacement for font rendering library and some very good info about texture packing, which could decrease loading times and memory usage for animations using spritesheets;
- optimized and cleaned shader code and tried to use them for some more advanced stuff we would like to implement at some point, like custom image blend modes;
- coded some other stuff... too minor to remember and list them.
Well, I think that's about it for this update. Not sure what I should tackle next, to be honest. Amount of places needing error handling added or improved is staggering, but I still haven't figured out what would be the best way to deal with it - got some ideas, but nothing solid. Window class implementation is closest to the finished state than anything else from the list, so it would be another good pick. Remaining required features, however, will be a major pain in the butt. Anyway, I've updated TODO list a bit, so take a look if you're interested.
Things recently done to make URGE production-ready:
- [REQUIRED] figure out directory contents handling;
- [REQUIRED] recode game fonts processing;
- [REQUIRED*] code shader version of Viewport's color, flash, and tone handling;
- [OPTIONAL] fix most noticeable italics font issue;
Things still to do to make URGE production-ready:
- [REQUIRED] finish implementing Window class - too much convoluted work to convert used vanilla windows to my own Window ruby implementation;
- [REQUIRED] design/implement creation of encrypted archive(s);
- [REQUIRED] overhaul error handling, reporting, and logging;
- [REQUIRED] fix whatever critical errors we encounter;
- [OPTIONAL] implement system fonts handling;
- [OPTIONAL] fix remaining font issues;
- [OPTIONAL] test and, if possible, implement game controllers support;
- [OPTIONAL] finish implementing scene transitions - for now Viewport objects are used for this purpose even in RM, so it can wait;
Malise Material Conversion to Redshift
Alright, back to the Maya/Redshift conversion!
Long story short: we need to recreate our Iray materials for the existing characters in Redshift. Skin, hair, the whole shebang. That’s not as scary as it sounds though, and as you can see I’ve already accomplished that with Malise (even while having to learn the software at the same time). Since they are both physically based rendering engines it’s largely a job of acclimating to the differences between all the knobs and gizmos and doing the grunt work :D.
It’s been a steep learning curve for me since Maya’s material editor is the first node-based one I’ve used. It’s far more flexible and powerful than Daz though, and now that I’ve figured out most of the tricks for the difficult stuff the other characters should go more smoothly. Close-ups should be a major improvement over Daz since Redshift is designed to handle much higher resolution textures, and with Maya I can and have taken the opportunity to layer in detailed tiling textures, as well as procedural effects to improve lower resolution stuff. Here are some examples of the improvements I made to skin while converting Malise’s materials:
Here are some close-ups of some of Malise’s props showing the vastly improved details I was able to add:
The big jobs were definitely skin, eyes, and hair. I easily spent the majority of a week on skin. Redshift’s subscattering model is different than Iray’s, and it took me a good while to get a good match. I then spent two days on eyes, three on the clothes and three on hair. Polygon hair is, it turns out, Redshift’s major weakness. Since Redshift was designed more for films, it was designed around the more detailed fiber-based hair. This became the cause of a big setback late last week.
Basically, Redshift can only see 16 layers of refraction/opacity deep. This means if you have 20 panes of glass it will stop calculating refraction after the first 16. Polygon-based hair uses texture opacity, and ours happens to be way more than 16 layers deep. We have however come up with a workaround that looks nearly as good as Iray did. The big difference is that you can see more hard edges – especially on stuff like bangs. Very luckily however, Redshift is redoing their depth tracing in 3.0 (hopefully available this year) and will allow for unlimited depth, at which point we'll make necessary improvements. Basically, crisis averted.
Anyways, even now I still have some work to do on her suit, but the majority of the work is done.
TK’s Maya Rigging Progress (Week 1)
TK here!
So, we decided to transition fully to Maya, and just use Daz Studio for creating the characters. This is quite the challenge for me! Character rigging is typically a specialized job, and I'm not too skilled in this area. Lots of heavy math, programming and scripting go into creating a good rig. Having a good set of controls on top of a skeleton is super important for animating and posing 3D characters. There are a number of available scripts out there to help here, but I had two major criteria. First, the initial skeleton and skin weighting must be kept intact or have a way to come back to the start once the control rig is created. Second, the initial geometry needs to be untouched so we can continue to add morphs to bodies and clothes coming from other Maya scenes or out of Daz. Many of the available scripts will botch #1 in one way or another; they're mostly designed for people with little to no experience and assume that the user doesn't have a skeleton bound first.
Eromancer had previously found a "Daz to Maya" script that I thought was mostly useless for our needs; it automates setting up a character for Maya's HumanIK middleware. Turns out it's not that bad, good even! It's not as fully featured or customizable as other rigs out there, but it takes a huge weight off of me. After some poking around I found that it meets my two basic criteria without problem. It's also a step up in comparison to using Daz for posing.
HumanIK doesn't handle faces or come with preset controls for some of the basics, however. It's handy to have a single slider that poses a hand into a fist or sets a facial expression for example. And that's what I'm working on at this moment. In the end, I'll have a custom script that can import a character of any shape and size from Daz and do all the setup through HumanIK, face control, additional rig controls, and whatever other small hurdles I might come across from here to there.
TK’s Maya Rigging Progress (Week 2)
Ero here! TK achieved a hell of a lot in his second week of working on rigging. Character rigs are working in Maya, and he spent an evening teaching me the basics of how to use the new one for Malise. Some other major milestones include:
- Functional face rig with advanced eye controls
- Joint corrective morphs for Genesis can be brought over and automatically hooked up. These are automated shape edits used to fix geometry to prevent clipping when joints bend.
- Grafts work! (grafts are additions/replacements for Genesis’ geometry – we use these for the nipples/genitals)
- We can convert poses from Daz to Maya (this one’s huge, I didn’t expect this)
Here’s a full update from TK!
More Rigging! I've learned a lot in the past couple of weeks and now have a functional facial rig that I can attach to any character we bring to Maya from Daz. Matrices just kicked my ass, but I have a much better understanding of most of the workings of them, like how to get world/local/parent/parents parent translations and do things with them. Getting the eyes to do everything I needed them to do proved quite the challenge, and I'm happy that I've gotten that working. Remember, I'm primarily someone who does models and textures; big 3d maths are scary!
A quick rundown on the eyes:
- Eyeball controls on the face and out in the world, we can use either.
- The world look-at controller can follow the character's root or the head via a switch.
- Master and individual controls for the eyelid sub-bones, we can use either.
- Eyelids will follow any of the above appropriately when the eyes look around, this deforms the skin around the socket as well.
- And all this wrapped up into a rig that I can place on any character. This is a big one for me.
Now that I have a prototype worked out, my next step is to write up a script to automate the whole attachment process during the character setup stage. I've been meaning to do some real work in Python for a long time now, this will be a good chance to do that. Meanwhile, I've only spared a little time to catch up with the animation toolset in Maya, and a script suite called Red9 that looks like it will be very helpful later down the road, but these are minor things that will go quick once we come around to it. First, we'll be bringing in all the existing poses from Daz, and that's simple at least- at least mostly. Apparently there's some "forearm rotation fix" that sometimes needs to be applied to Daz poses. Eromancer had mentioned it in passing but I haven't looked into whatever that is. You can see the bad rotations in two of the poses linked above, but it's the minor stuff like this that happens daily.
Maya Conversion Progress Summary / TODO List ***
Still with us? I know we’ve thrown a crap ton of information at you in one go (especially our $3 patrons, sorry guys :D), but here’s where I wrap it all up.
As you can see here we’ve got most things working. This was a full test run of the entire conversion procedure that we’ve put together over the past 24 hours. The rig works very well – even better than Daz’s in a lot of ways. Some of the major remaining issues are:
- Remaining subdivision / HD morph import issues for rigging (notice there are no folds/details on the gloves or boots here, and most things are kind of low poly). Also the clipping you see in areas is a function of this.
- Figure out how to convert poses for non-body parts (Hair, etc.)
- Figure out shape discrepancies. After rigging, Malise’s face and body don’t match entirely, and we aren’t quite sure why yet (this is actually a new one we found while running this test over the last couple of days).
- I still need to do thorough testing with the face/eye control rig, which I haven’t had a chance to use yet. It’s not hooked up in this test.
There’s also a big list of less apparent issues that TK is still working on. He also intends to automate more of the rigging conversion process. In the mean-time I have plenty of stuff to do on my end.
I spent a lot of this past week converting our battle/portrait lighting rig to Redshift and figuring out the best render settings for portraits. We’re now using direct lighting instead of global illumination, which is much more compatible with Redshift’s more advanced features, but it is also noisier. Ubercharge therefore had to come up with a clever solution to get the noise down in our renders while maintaining our high dynamic range lighting. It works like a charm! I’m also happy to say after this test that Redshift is very fast, starts nearly instantly, and is generally a quality of life improvement over Iray. I’m confident I’ll be able to cut our character render times down significantly. Anyways, here’s my list of tasks for at least the next couple of weeks:
- Calibrate the lighting rig to match our rig in Daz
- Set up the light group render passes in Redshift for exporting to our post-processing/lighting automation network
- Make any necessary adjustments to our automation network
- Begin converting materials for Neon and the other characters
- I will probably need to convert the police enemy to Genesis 3, which is unfortunate but is something we figured may need doing at some point anyways.
There’s a bunch of organizational stuff we’ll have to pause and figure out at some point. The new art style was one thing, but a full conversion to Maya means we’ll have to really reorganize our file structure and cloud drive layout.
Anyways, I’m glad we were able to have some results ready for you guys. I realize it can look like we’re treading water from your perspective but doing away with the training wheels is the best tech decision we could make at this point if we want to continue to evolve and make this project manageable!
Thanks again for your continued support and for sticking with us as we push forward :D.
Comments
I Love Being Anonymous, isn't it great
It's too bad that you went for leather instead of latex. I don't know about most people, but a latex suit would have been much preferred over leather. Just my $0.02 worth...
Definitely not that soon :D. I'm kind of hoping to be back to where we were in Daz and ready for the Enty launch in or before August. This would mean having Malise/Neon/the police enemy and all their poses redone for Maya, Malise and Neon's armor damage completed, and all our systems for rendering/lighting/automation working with Redshift (the new rendering engine). This would basically be Battle Test 4 -- which would include H artwork.
Yep, a big advantage of 3D art is that once you have a solid rig and models you can build off of your previous work. It's a big tougher to organize things like poses or facial expressions in Maya than in Daz (which was designed around organizing content like that), so we have to make our own custom systems for it. As we flesh these out things will become more streamlined than they could have in Daz.
I was wondering, since i have 0 knowledge in 3D art, after you guys complete all the character models, will that speed up the process of making new content? For example if you have Neon and Malise' models done,will it be easy to just change their current facial expressions and pose to add them in different aspects of the game ? (mainly in H content but I'm talking in general :D) I'll give another example with Illusion games (we can hardly call them "games") but there are a lot of poses and stuff ready in the game that you can change their clothes, poses and other stuff such as facial expressions in the game and you can make custom ones in the studio and it takes about few minutes/hours to do so.
There are actually some unintended differences on her face in that pic -- same with body shape. TK has since fixed some of them, but the conversion to Maya's rigging is really complex and there are lots of little things to be sorted out. These all have to go in a custom script TK is working on as an addendum to the commercial scripts we are using for the conversion process.
myself yourself
hum now that i was paying more attention to her face up close, confess for some reason i dont like something on malise's face, hum did u guys changed something on her face? that nose looks too tilted upward and look like the typical girl that had plastic surgery over her nose... but everything else seems great, love the progress so far :D
Friends don't let friends drink and program...although in some cases it leads to some unexpe...no nevermind, that didn't end well with the drunk welding and nearly lighting myself on fire.
Renaikaze
I understand your point and glad that you making progress. It looks better and better But i just want to know around when to expect 0.06? July? August? You can even lie to me about it :) i just wanna have a goal in mind XD
Carter Parks
Good to see some solid progress, and happy to hear that we're one step closer to the next full release. (And TK, YOU'S AN OLD MUTHAFUCKA. GO GET SOME BOOZE TO PASS THE TIME, BUDDY. Joking, of course...you're still young as far as AARP is concerned.)
One update closer to V0.06, one update closer to Malise's Valkyrie outfit.
We can't make forward progress without proper and reliable fluid rendering, and Daz / Iray aren't up to the task (unless we use either very time consuming techniques or the super bad stuff you commonly see in Daz art). The new art style requires that liquid be done in 3D if we want to automate lighting, meaning we can't paint it on and call it a day like we did with the old art style -- and we'd like to try to match the quality of the hand painted stuff. Once we're set up in Maya / Redshift we'll have lots of options and be able to continue on v0.06.
WE NEED THE GAME <3 can we go back to the old time? where every mth we get an updated game ;_; coz im running out of money ..... to get the full game when u guys are done... every mth - $10 for me :(((( been supporting for 6mth-ish? <3 you still !
Renaikaze
That's all awesome and all. But can someone tell me ETA of 0.06?
Since it's vector based it will be selective, not just a big smear on the entire image :D.
Faithel
You guys are amazing. What a huge update! GJ!
SomeSortOfMillu
i hope that Motionblur isn't gonna be too strong
I actually started reading this now because i didn't have a lot of time earlier and was only scrolling to see the pictures and i gotta say ... HAPPY BIRTHDAY TK I'm glad you're part of this project :D
Can't wait to see the bottom as well :D
Man that Onyx top is SOOOO FREAKING HOT i love it, :D but still not hits for release date ._. cant wait !
