‘We were a bit too ambitious’ – how AMD is upping the game with FidelityFX Super Resolution

AMD launched its exciting new FidelityFX Super Resolution (FSR) technology in June, promising to bring PC gamers improved performance while running graphically intensive games.

FSR achieves this by upscaling games in such a way that while they run at lower resolutions, they look incredibly close to how they would if they were running at higher resolutions, such as 4K. While superficially this sounds like a rival to Nvidia’s DLSS (Deep Learning Super Sampling) tech, there are quite a few key differences, including the fact that it’s open source, multiplatform (able to run on AMD and Nvidia GPUs) and doesn’t use machine learning (AI).

We chatted to Nick Thibieroz, Director of Game Engineering at AMD, about FidelityFX Super Resolution and how it came about.

The birth of FSR

TechRadar: The FidelityFX Super Resolution announcement has got a lot of us gamers very excited, but how did the idea of FSR come about, and what spurred the team at AMD to come up with the technology?

Nick Thibieroz: I think what we've seen the last couple of years, is that games tend to push out a specific tech that are quite costly, in particular raytracing. So, ray tracing itself is obviously a quite interesting piece of technology when it comes to pushing realism in games, but it comes at a heavy performance cost. And this is true for both Nvidia and AMD GPUs so far.

So, we wanted to provide a solution that would still allow gamers to play at high resolutions, but without having to incur the additional costs of raytracing workloads, that would be required, obviously, from having to add more pixels. The idea is that you still do your ray tracing operations at a fixed resolution, and then you've got upscaling on the back end to bring it back to a high-quality high resolution.

But also, another argument is that this is not just about ray tracing, this is true of any graphics techniques. But, obviously, ray tracing is a significant factor in our decision to research this kind of tech. What we've heard from game developers is that they are now more interested in rendering 'better' pixels, compared to more pixels. That means it makes more sense to really know what your GPU budget is, and focus the GPU performance budget on making the best pixels as possible.

It also kind of standardized what they're doing. If they try to render all the effects targeting different resolutions, the task becomes more difficult. However, if they can say ‘we are going to target 1080p’, for instance, ‘and a minimum frame rate of 60 FPS’, something like that, then it allows them to do a better job of managing that GPU budget better.

We are seeing this already on the consoles. On the PC, I think we're getting to the same kind of space, right, where this unification will be very desired. So, we are trying to give game developers what they want; a solution where upscaling is available everywhere, right? So that they can basically work on rendering those better pixels.

TechRadar: So, part of the inspiration cam from talking to game developers? And AMD thought ‘oh, there’s something in this?’

Nick Thibieroz: That's exactly right. Multiple times a year, at least prior to COVID, we used to visit game developers face to face, and then sometimes we'd bring our senior architects as well. These meetings are great, as they're occasions for us to really pick the brains of the game devs in terms of the current technology that they're developing. But also, it helps us find out how they want to use future hardware. So, we always do this exercise in terms of picking their brains, understanding what they want to see from us, as a GPU manufacturer. And, we kept hearing from developers that they want to see those better pixels, but having to cater to multiple resolution and platforms is more difficult, right? So, for them, it’s easier to upscale on the back end, while providing those better pixels. So yes, that data with game developers is definitely a huge factor in our decision to research and publish this tech.

TechRadar: So once you saw the desire from the developers, and also from gamers as well, who want to take advantage of these advanced graphical effects or visual fidelity, what was the next step  to coming up with FSR?

Nick Thibieroz: From that point onwards, we had to basically think 'okay, so we want to do this, but what are the goals? What are we looking for? What is it we want to enable, how do we want to make developers' lives easier?' But also, how do we want to help our customers? So, this is how we came up with the idea.

We're doing 3D effects, we're doing GPUOpen, which is pushing for open standards. So, we had multiple goals in mind. Also, we definitely wanted something that was available for any platform. We didn’t want to go the route of proprietary and closed technology, because we knew that would limit adoption, and more importantly, it would be contrary to the whole philosophy of GPUOpen; the best way for the industry to be pushed forward, in terms of innovation, is by sharing knowledge. And that means open-sourcing code, allowing others to understand the code from looking at it, allowing them to optimize it, and also making the code as accessible as possible, to allow people to do the best job of integrating that code. So, that was a major goal, obviously, being open.

Also, we wanted to provide a solution that works in a cross-platform manner, so we did not want to make it too high in terms of required functionality. And that also meant we had to have something that is fast, right?

In fact, we are very proud of the overall performance in terms of the overhead. The actual cost of running FSR technology is very small, really, for what it does. And, ironically, I don't think this has been talked about enough in those press reviews. The tech is great, obviously. But what it does, and for the cost [when it comes to PC resources], I think is very impressive. So, we are particularly proud of how performant it is.

But, that performance meant we could allow lower-end hardware to also benefit from the tech. And, the reason why we wanted this is that we're very much aware that the GPU market, currently, is... difficult in terms of prices, in terms of availability, and other things.

So, by providing a technique that everybody could use, and not only our customers, but customers from other companies, then we basically help every player out there to get access to better visuals and better experiences in their games.

The ease of integration was another goal. If you're targeting multiple platforms, you're going to want a piece of tech that is as easy as possible to integrate, right? So, we found that FSR ticks that box. We're providing the same code for DX11, DX12, and Vulkan, along with game engines like Unity and Unreal. So, regardless of the API or the engine you use, it becomes actually very easy for game developers to integrate the tech.

So, the technology itself had to support what we call fixed scaling and arbitrary scaling. Fixed scaling means your first resolution is basically fixed to any given resolution, and then you scale it to any given output resolution. Arbitrary scaling means the scaling factor to apply can be anything you want between 1x and 4x.

The reason why this arbitrary scale is important is because there's a piece of tech that is used in games called dynamic resolution scaling. And that goes back to what I mentioned earlier about better pixels. The idea is that you've got a fixed performance budget. Say you want your game to run at 60 FPS. So, the game will automatically adjust the source resolution, so that you're always within that 60 FPS. If your game starts requiring too much GPU cycles, and it goes basically below 60 FPS, then. automatically, the engine basically reduces the size of your source input. And obviously, this is where FSR can come in.

So, regardless of your source input, your destination resolution is the same, and we wanted to support this very important feature.

And, finally, we wanted to show the tech actually being used in games. So, instead of releasing the tech directly on GPUOpen, we wanted to make sure that we worked with a few game devs, and have them actually test the technology themselves, integrate it themselves, and actually release patches for their games with our technology so that people can actually witness it first-hand, as opposed to just being a piece of tech that's on GPUOpen that really no one can test beyond the sample that’s provided.

TechRadar: So, when you showed game devs what you’d been working on, what was their reaction like?

Nick Thibieroz: Yes, that's interesting. Quite a few devs were surprised that we went with a spatial-based upscaling. So, FSR is pretty special in the sense that it does not rely on previous frames, or motion vectors, or any jittering of the frame to produce the upscaling technology.

That’s got pros and cons, right? The con is obviously that you've got less data to work with. The pro is that it's also a lot easier to integrate, which means any game can be integrated. You do not need to have TAA (Temporal anti-aliasing) already integrated in your game to take advantage of it. In fact, any anti-aliasing technique can work with FSR as long as there is some anti-aliasing. So, sometimes [the devs] were surprised that our technology, at least our first foray into FSR, was spatial-based.

But, they're also pleasantly surprised to see the quality on offer for the performance cost. So, that was also quite a surprise that they could get this level of quality for just an upscaling technique. So, in that respect, we were pretty happy with the response.

Standing out from the crowd

TechRadar: FSR is also more software-based rather than requiring particular hardware requirements, like Nvidia DLSS. So how did you settle on that kind of that method?

Nick Thibieroz: We wanted something that was accessible for multiple generations of GPUs, including past GPUs, to be able to run it. So, we could not set the bar too high. As to why we chose a spatial technique instead of anything else, that is a longer story that I can explain.

Essentially, once we decided that we wanted to develop an upscaling technology, we actually had multiple teams at AMD researching what kind of technology we could provide that ticked all the boxes I mentioned earlier, in terms of goals. So, we had several teams working on this. And then one of those teams was actually led by Timothy Lottes.

He's quite well known in the industry, he now works for Unity, and he actually presented on FSR a few weeks ago at SIGGRAPH. But Timothy is known for FXAA (Fast approXimate Anti Aliasing) and CAS (Contrast Adaptive Sharpening) and LPM (Luminance Preserving Mapper), like a bunch of technologies. So, essentially, he led the research for one of the upscaling teams at AMD.

And, It was a long process, right? And one day, we ran the review process, where different teams provide the results of their current tech at regular intervals. And we basically look at the results, and performance and cost, all those things. And then we'd decide on the next steps, and whether to prioritise a certain tech over a different way of doing it.

So, one day, Timothy provided a version of upscaling technology, and we knew straightaway that we had a winner, because we saw the edge quality, which is very important for resolution, right? The quality of the edges produced has to be similar to what you get when you're rendering natively at high resolution.

And when we saw the quality of the edges produced by Timothy’s algorithm, it was actually looking as impressive as another technique that we were also developing in parallel, but that was a lot more costly, from an implementation and from the GPU cost perspective. So, we knew we had a winner there.

From that point onward, we decided, 'okay, we're going to go with that.' And then we refined the algorithm, and we shared an alpha version of the tech with a few selected close partners, then we got feedback from them, we helped them in integrating it, and that's how we ended up producing the tech.

But, initially, we did not start the project saying, 'okay, we need to do spatial.' No, we looked at different solutions, and to be honest, we are still looking at different solutions. But we felt we had a breakthrough with a spatial solution, especially with regard to performance and meeting all those goals. We feel the balance is just right for FSR, so we went with it.

Facing challenges

TechRadar: So, during development, what would you say some of the biggest challenges were that the team faced when creating FSR?

Nick Thibieroz: I think the biggest challenge was how ambitious our goals were initially, right? How do you provide something that looks great, is fast, works on every hardware, or as many possible platforms as possible? That is also easy to integrate? I think we were a bit too ambitious to be honest with you initially.

So, trying to fit all those goals into the technology we ended up with, I think, was probably our biggest challenge. We looked at different solutions out there, and we saw existing simple upscaling solutions, and they did not provide the quality or performance we were after.

And then, when we looked at more complex solutions, like ML-based or temporal, they typically required a high level of performance, or maybe they were too complex to integrate, and we wanted to avoid that, because that was not our goal for at least the first solution we wanted to put out there.

So, yeah, I think finding the right balance between all those metrics, all those goals, I think was probably the biggest challenge when we created this tech. But I'm very happy, as I said earlier, with the balance that we were able to achieve.

TechRadar: So, now that FSR is out there, how do you feel the gaming community has reacted? How did it make the team feel?

Nick Thibieroz: We've been humbled by reception, to be honest with you. I mean, we knew we had a winner on our hands, when we saw that edge quality for the cost... that is really cool. So it's one of those breakthrough technologies, like, for instance, FXAA [an anti-aliasing technology]. FXAA is at least 10 years old, right? It was a fast and cheap way to do things. And at the time, it was ubiquitous, it was used everywhere. It was a very, very attractive piece of tech.

I would actually make a comparison between FXAA and FSR, in terms of its adoption rates, and its popularity with game developers. After all, we should not underestimate the importance of cross platform compatibility and ease of integration. So, we've seen all those devs, I think there's more than 40 devs that have already agreed to support FSR. There's a lot more, obviously, that are also planning to do so.

So yeah, it's been great, really. We're still helping devs, if they need our help, to integrate [FSR], but again, it's so easy to do. And we've got all the documentation now on the website. What we're seeing is devs get early access to implement FSR in their games, and it looks great already! We barely have anything to tell them.

We were also humbled by the reception from the devs in terms of their comments to us. I think there were a couple of people that were happy with their own solutions, which were temporal, and that's fine. I'm absolutely okay with that. But others, you know, that didn't have anything, so they were very happy with having something so easy to work with. 

And, the fact that the tech is now obviously used in the Xbox development kit is great. It means that as a game developer, you can actually have the same piece of code running on both Xbox and PC, which makes your porting task a lot easier. So, lots of happy faces with game developers, and it's been great to see that kind of reception.

The importance of open source

TechRadar: One of the elements that got the most praise, it seems, was the fact that FSR is open source. How important to the team was it to make sure FSR was as open as possible?

Nick Thibieroz: Well, I was certainly pushing for this. GPUOpen is handled by myself and my team at AMD. So, it's a project that I've been working for a very long time on. It's – I would say – a philosophy at AMD where we believe in industry standards and open source for the reasons I mentioned earlier: integration, sharing of knowledge, innovation, and other things.

So, it's been great to see the positive reception due to the tech being open source. The sharing of knowledge, and I want to emphasize this one in particular, has always been at the center of the game and graphics development industry.

That’s why open source was really important to me and my staff. It was something that I pushed on. The tech we ended up with did not require any specific hardware, and I wanted at the base level to be very accessible. So, we did not need to make it closed source anyway. So yeah, it's really something I personally pushed on, but I was very happy to see that AMD's management was really on board with that. They understand the value of open source.

TechRadar: The fact that it works on older GPUs, as well as graphics cards from your rivals, is a bit of a fresh air, compared to the competition.

Nick Thibieroz: Our approach is very different from other approaches out there, that's for sure. I won't comment on what our competitors do, but as far as AMD is concerned, we believe that open source is, whenever possible, the way forward.

TechRadar: We've talked about FSR coming to Xbox, and I believe there's a PlayStation 5 game that’s going to take advantage of it as well. And then, do you think there's scope for FSR on devices like the Steam Deck, which is obviously something that’s got a lot of people talking about at the moment?

Nick Thibieroz: Yeah, I think at the end of the day, FSR is an open-source solution, so it can be integrated by anyone onto any platform that supports the base FSR technical requirements. So, I can't really speak for the platform owner, it's their decision, or the decision of the game developer, whether FSR is a suitable tech for that platform [or game]. I would say, technically, there is nothing that would prevent it from happening. But it's really [Valve's] decision.

Looking to the future

TechRadar: So, what does the future hold for FSR? What would you like to see happen next with the tech?

Nick Thibieroz: As I said earlier, I think it's been humbling to see the meteoric adoption of FSR in so many titles... that's been really great. We are continuing to help game devs [and] give them pointers on how to integrate it in the best way possible, so, that work continues. And, obviously, my team is at the forefront of those efforts.

But, it's a bit too early to talk about what is to come, of course, right? But what I can tell you is that AMD is still actively researching, developing and improving graphic tech. And that includes upscaling. It's an interesting market now, because you've got AMD, Nvidia and Intel providing their own upscaling solutions. So, I think it is safe to say that upscaling is here to stay.

TechRadar: And with FidelityFX Super Resolution, you believe the route you’ve taken is the right one?

I would say given the adoption, it's definitely a good route. We've seen integration of the tech by many game developers... and, again, we are not averse to people using different technologies! Obviously, we have a preference for anything that is open source, because that is the right way forward for innovation and so on. And, we are working on other solutions, as I hinted earlier. But, FSR, in my book, is a very good balance between all those factors. It's possible that even if (or when) AMD releases a future version of FSR, it doesn't mean that the first version of FSR will be completely superseded. I think there may be a case where people will want to use FSR 1, and others will want to use FSR 2, and so on. But again, it's too early to talk about this.

• Welcome to TechRadar’s PC Gaming Week 2021, our celebration of the greatest gaming platform on Earth. Despite the global pandemic and ongoing GPU shortages, PC gaming has never been more vibrant and exciting, and throughout the week we’ll be reflecting this with a selection of in-depth articles, interviews and essential buying guides.