Heavenly Sword HDR Interview (with some additions)

Ansatsu

Power Member
A entrevista já foi aprovada pela Sony, por isso volta aqui e tem alguns temas acrescidos.


When I joined the PSINext forums two years ago (it was called PS3Insider back then), one of my reasons for doing so was to glean knowledge of the Cell architecture from several individuals that seemed ahead of the curve in their understanding of the Broadband Engine. Marco Salvi (aka nAo) was one of those individuals.

Thus it seems only fitting that two years later, here I am asking him questions once more; this time in a more formal setting.

The topic of this interview is the custom high dynamic range lighting format Marco has helped to develop in his brief time with Ninja Theory. This rendering format - NAO32 - is one of the key reasons why Heavenly Sword has emerged as one of the most beautiful games to have made an appearance at this years E3, winning spots on the top ten lists of several publiations.


PSINext: Marco you've been with Ninja Theory for almost a year now, is that right?

Marco: I moved from Italy to the UK last September, so I've been living in this country for something like 8 months now (wow, time flies!) I'm not the biggest gamer out there; I work in this industry because I love cutting edge technology and computer graphics. Moreover, I have a passion for CPU and GPU architectures, so it was a natural thing for me to start working in games development.

PSINext: What inspired you to move to England and get involved with Ninja Theory in the first place?

Marco: After having spent 3 years in Milano working on a PS2/XBox title, I was kinda bored and started thinking about going abroad and trying something different. At the same time, a well-known and respected bloke called Dean Calver (lead programmer @ NT) asked me if I wanted to send over my CV because they were looking for someone that could join them and work on their in-house PS3 engine. I was so excited at the idea that I could not believe him... then quickly came the realization that it was an exceptional opportunity for me. I had an interview in Cambridge and 3 weeks later, I relocated to the UK and started to work on Heavenly Sword.

PSINext: High dynamic range lighting - or HDR - is a term we're starting to hear more and more of these days. Can you explain to us exactly what HDR is, and what it means to people sitting in front of their screens playing a game?

Marco: Yeah, it's a very common term nowadays and it's often misused.

When we render an HDR image, we are basically storing all of the information we need to represent the amount of light that passes through every pixel of our picture, so that we can capture a scene in all its color and luminance-range richness. Unfortunately, current low-cost display technologies (common LCDs, CRTs, etc..) can't properly display an HDR image, so we need to go through a process called 'tone mapping' to remap our image to an LDR (Low Dynamic Range) image in order to display it on a screen.

Tone mapping can also be used to simulate the way our eyes slowly adapt to different light conditions. In the end it means a better visual experience for gamers, although it will take time for the industry to get it right.


PSINext: What are some examples of scenery within a game that benefit from HDR?

Marco: Compared to LDR rendering, every scene that requires a very low or very high global luminance image, or an image with a high contrast ratio, would benefit from HDR lighting.

It's a way to avoid globally saturated images where everything comes out too bright or too dark and the viewer loses all that precious lighting/color information that enriches an image and makes it believable.

Furthermore, a specific tone mapping operator can give to a developer an extra tool to improve story telling, because it can be tweaked to simulate particular behaviors of human vision. It's important to understand that HDR is not just about nice bloom effects - it's much more than that. There's no such thing as LDR imagery in our every day lives.

hs_hd_4.jpg


PSINext: People have come to associate high dynamic range lighting with on-chip hardware support for either FP16 or FP32 HDR rendering. How does Team Ninja implement NAO32, and can it be considered 'real' HDR?

Marco: The FP16 and FP32 rendering formats give a developer the opportunity to collect per pixel information (respectively 8 and 16 bytes per pixel); hence they easily enable us to render and to store an HDR image. Unfortunately, these framebuffer formats are inherently slow because they require more memory bandwidth and increased memory space: an FP16 720p image with 4X anti-aliasing requires about 30 MBytes of memory!

At the same time it's important to understand that it does not matter how we store our HDR images so long as we find a way to encode them without losing too much information.

The RGB color space is not very efficient at encoding HDR images, so after a bit of research we found another color space that is far more efficient at representing HDR images. Its name is CIE Luv, and it splits a color into 3 components: one is not normalized and represents how intense a color is (luminance), while the other 2 components are normalized between 0 and 1.

Gregory Ward, a pioneer of HDR imaging, exploited this color space many years ago to store HDR images in a file format he called LogLuv, so we built upon that work and we customized it to our purposes.

PSINext: So NAO32 is a means for you to preserve memory while at the same time retaining image quality, is that correct?

Marco: Correct. The main idea behind NAO32 is that we want to trade shading power to regain memory space and bandwidth (very precious resources on a console). So instead of encoding our HDR colors into a FP16 or FP32 frame buffer, we devised a scheme to use RSX pixel shading units to convert an RGB color in a CIE Luv color that only requires a common RGBA8 frame buffer (4 bytes per pixel, half the space of a FP16 pixel) to be fully stored.

The quality of this format is really outstanding. Even if it uses half the space/bandwidth of common HDR rendering solutions, it really makes no compromises at all in image quality.


There's no magic here: HDR rendering costs are shifted from memory to shaders, and so our shaders are a bit longer now (between 3 and 5 cycles). We believe it's a very good trade-off. Furthermore, it enables HDR rendering and multisample anti-aliasing on GPUs that do not natively support AA with floating point render targets such as FP16 and FP32.

We also developed a faster 3 bytes per pixel version called NAO24 (predictable, isn't it?) that supports a narrower dynamic range with less accuracy. And although the quality was quite decent in most cases, we decided against making any compromises, and so in the end we did not use it.

My final answer is totally positive: NAO32 can be considered a real HDR format.

hs_hd_6.jpg


PSINext: The advantages of reduced space and preserved quality seem like they would have merit in a number of environments. Do you think there may be a place for NAO32 on the desktop, or even on Microsoft or Nintendo's offerings?

Marco: Dunno about Nintendo's offering, but it might have merit on Microsoft's console if developers wanted something that takes the same storage space as an FP10 render target, but with a much higher level of quality. NAO32 on Xenos would cost developers shading power relative to FP10, however, and they would lose the ability to use the eDRAM for blending as well. So at this time, I believe something like NAO32 makes more sense on RSX than on Xenos.

PSINext: We've spoken here about a number of NAO32's advantages. Are there any notable drawbacks?

Marco: Yes, there are drawbacks too; it's not an all-win situation. GPUs usually support hardware blending in RGB color spaces, hence hardware assisted blending is not going to work on a NAO32 frame buffer - it would produce incorrect results.

There are various ways to overcome this limitation though, such as by emulating blending operations in a pixel shader, or performing blending on a FP16 render target and then composing the result… or even just blending in LDR in a common RGBA8 buffer.

It would be nice to have a GPU that natively supports a CIE Luv frame buffer though.

PSINext: If I may ask, what originally inspired the name NAO32?

Marco: I didn't name it, actually Dean did on Beyond3D. You should ask him! Internally, we used to call it "the funky color space."

PSINext: That's right, I remember that now that you mention it. NAO32 is the better name, for sure.

Straying from HDR for a bit before we finish up, now that Sony has announced the hard drive to be standard in every console, how do you feel this might effect game development in general, and Heavenly Sword in particular?

Marco: I believe it's a big opportunity for all developers to make better games. This generation the ratio between available memory and optical disc read speeds is much higher than what we had in the previous generation, and a standard hard drive is going to help us reduce loading times and give gamers a better experience.

Regarding HS, this announcement is not going to change our plans.

[It has been confirmed that Heavenly Sword will be making use of the hard drive]

PSINext:
After getting to play the game at E3, it's clear that a number of intensive effects such as HDR - courtesy of NAO32 of course - and full soft shadowing are in place. What can you tell us about the levels of anti-aliasing and anisotropic filtering, if any, Ninja Theory is utilizing for Heavenly Sword?

Marco: Yep, shadowing is completely dynamic and everything can cast/receive shadows. Soft shadows are achieved taking 12 jittered samples per pixel. Antialiasing is set to 4X (multisampling), but quality wise is not as good as it could be; we need to work on it, and hopefully it will improve over the next several months.

Anisotropic filtering is being used on some specific meshes (floors, walls, etc...) and AFAIK is set to 8x + trilinear.

PSINext: Have any internal resolution and frame-rate targets been set yet for HS?

Marco: Our target resolution for Heavenly Sword is 720p with 4x MSAA, which we've already achieved. The frame rate target is not something completely set in stone at this time. Though our E3 demo was running at over 30 frames per second, I'm willing to bet the final game will run at 30 FPS. Hopefully this will allow us to push even more effects on screen.

PSINext: As previously discussed, beyond it's high quality one of the primary reasons for the use of NAO32 is that it saves bandwidth in a bandwidth-hungry environment. In the future do you feel RSX will be at a disadvantage to Xenos when it comes to framebuffer effects due to the 128-bit bus and lack of eDRAM?

Marco: Not at all; in fact for many framebuffer effects I believe RSX will have an edge over Xenos. Don't want to go into details, but let me just point out that RSX is connected to two seperate buses, not just one.

PSINext: After speaking with your Ninja colleagues Dean and Mike on the floor of E3, I've been told that Ninja Theory is presently using SPEs for tasks such as sound, physics, animation, and more. Since you're working on the engine, going forward what sort of performance headroom do you feel is present on Cell?

Marco: I can't give any numbers that would make much sense, but I'm certain at this time most developers are barely using Cell's power.

We all have to learn how this machine works and how to get the best out of it. It will take time, but gamers are going to experience some truly amazing stuff.

hs_hd_9.jpg
 
Última edição:
Ou seja a ATI continua a inventar um HDR estranho em vez de irem pro FP32 standart... Mas se funciona e dá bom aspecto à coisa, não vejo porque não...
 
God_Lx disse:
Ou seja a ATI continua a inventar um HDR estranho em vez de irem pro FP32 standart... Mas se funciona e dá bom aspecto à coisa, não vejo porque não...

Pelo contrario. Este é um novo formato chamado NAO32 que á custa de alguma de perca de alguma performance pelo pipeline (para transformar RGB em CIE Luv) consegue ganhar no tamanho que cada frame ocupa na ram que é bastante mais pequena deixando mais espaço para texturas.

Ele diz que as diferenças de qualidade para FP32 sao pequenas, mas sendo o criador a falar da sua propria criação é natural que "puxe a brasa á sua sardinha".

Esta á daquelas coisas que nao percebo na consepçao de uma consola.... Pq por mais 30 euros nao tinham posto mais 256mb de ram para a grafica e deixavam de ter problema de falta de espaço depois para texturas e ainda nao era preciso perder performance no pipeline na transformaçao RGB -> CIE Luv.

Claro que nos mercado dso PC´s com o crescimento expoente da ram nas graficas esta soluçao nao é muito necessaria no futuro. (já há graficas FireGL e Quadro de 1gb de ram!)
 
DJ_PAPA ele diz k a diferença para FP32 é pouca, mas existe, logo inventou e perdeu qualidade... Sinceramente até acredito k seja uma coisa tão minuscula k n será visivel, mas seria preferivel nem sequer existir...
 
God_Lx disse:
DJ_PAPA ele diz k a diferença para FP32 é pouca, mas existe, logo inventou e perdeu qualidade... Sinceramente até acredito k seja uma coisa tão minuscula k n será visivel, mas seria preferivel nem sequer existir...
Mas atençao que o NAO32 é para usar na PS3 e nao na Xbox :P
Na Xbox usa outro tipo qq de HDR. Fala la no FP10, mas nao sei se é esse o standart.

Depois podem vir mais jogos com este HDR para qq plataforma.
 
Mas acho que o NAO32 é a melhor alternativa.

A 720p com 4xAA e FP16 ocupa 30 megas de ram, por isso eu nem imagino quanto não ocupa de ram o FP32. Neste caso a x360 tem que usar menos que FP16, pois tem 10 megas de eDram e não cabia lá tudo para terem o AA de borla.

E depois o NAO32 é melhor que FP16, e ligeiramente pior que FP32, e ocupa menos espaço k o FP16, logo acho que não existe melhor alternativa.


Claro que o ideal, era a Sony meter mais 256 de memoria Ram á ultima da hora, como fez com a PSP, em que passou de 8 megas para 32 megas e supreendeu toda a gente. Mas não me cheira que aconteça.
 
HDR já não é uma coisa propriamente nova!!!!

Mas as tecnologias usadas para o fazer podem evoluir ou nao?
Se desenvolverem alguma técnica que faça o HDR com mais qualidade sem grande impacto na performance é sempre bemvinda, além de que é interessante saber quais as vertentes que cada hardware gráfico vai tomar no cálculo do HDR.
 
Pronto, lá tinah de vir a pessoa k nucna ve nada de positivo em algo ;)

Não é apenas o HD que está a mais, a gaja tá mto mais detalhada e os graficos mais definidos.
Provavelmente ainda melhora até sair o jogo.

lá vem o crítico.. do "tal e qual"

Várias técnicas para fazerem HDR já existem.. até se falou que entre a XBOX e a PS3 a técnica irá ser diferente.. o tal nod32 e o fp32..

até a valve consegui meter um HDR numa 9800..se bem que esta já trás algum hardware nesse sentido..

Que haja mais detalhe é natural...

mas continuamos na mesma.. é uma foto..nem sei se é in-game.. portanto.. continuo com as mesmas dúvidas de dantes..
 
Mas acho que o NAO32 é a melhor alternativa.

A 720p com 4xAA e FP16 ocupa 30 megas de ram, por isso eu nem imagino quanto não ocupa de ram o FP32. Neste caso a x360 tem que usar menos que FP16, pois tem 10 megas de eDram e não cabia lá tudo para terem o AA de borla.

E depois o NAO32 é melhor que FP16, e ligeiramente pior que FP32, e ocupa menos espaço k o FP16, logo acho que não existe melhor alternativa.


Claro que o ideal, era a Sony meter mais 256 de memoria Ram á ultima da hora, como fez com a PSP, em que passou de 8 megas para 32 megas e supreendeu toda a gente. Mas não me cheira que aconteça.

Tas a falar de espaço na ram, espaço na ram e mais espaço na ram.

O problema do NAO32 é que precisa de perder algum poder de processamento para fazer o truque de passar de RGB para CIE luv e assim poupar espaço em cada frame.

Por um lado é bom (sistemas com pouca ram), por outro lado é mau pq tira poder de processamento.

Pode ser um formato a vir a ser utilizado nas consolas, mas nao nos PC´s normais. Ai a ram abunda e usa-se logo a maxima precisao FP32 para melhores resultados.

Essa tecnica pode ser usada em qq plataforma, mas nao parece que compense. Foi desenhada mais para a PS3. A Xbox tem um GPU totalmente unificado com a ed-ram e outros metodos de HDR podem aparecer para tirar maior rendimento disso.
 
Back
Topo