Image Formats & Contentful's Images API Podcast Interview

Allan White: Yeah, absolutely. I love images. And the Contentful Images API was one of my favorite things about Contentful that drew to me as a user.

Well, my journey with photography started pretty young. My grandfather was a professor and my dad was also into design as a young man, and photography and he had a Yashica 35 millimeter camera, an old analog one with a 35 millimeter lens when I was about 15, probably mid 1980s, started shooting black and white photography and taking it seriously. And I really got into photography did a lot of landscape photography as a kid, you know, admired artists like Ansel Adams and Henri Cartier-Bresson, you know, some of the great photographers. It’s something that led me to pursue design as a career. So I studied design in college, I have a Bachelor of Fine Arts in design, graduated mid 90s. And started my work in tech and doing a variety of jobs over the years. Images have always been a part of that.

Images, in fact, were kind of my earliest, like, serious sort of design discipline. My work in tech, you know, led me into questions of how do you get the most business value out of these important assets? How do you you know, and as a designer, I’ve been engaged with questions like how do you choose and deliver these images? How do you crop them? How do you choose them before maximum impact, right if you’re trying to move the lever On a particular, you know, you’re trying to persuade someone in a brand sort of moment, how do you select them? How do you alter them? How do you, you know, do all those things in Photoshop that I did kind of as a young man, but now in my later career, it’s kind of like it’s shifted to more of, how do we get the most business value out of them? And part of that value is, how do you deliver them fast? How do you have big, gorgeous, beautiful images that tell your brand story that show your brand promise that show off your product, that kind of convey the message that your organization is trying to do, but do it in a way that’s highly performant and just optimal is the right sweet spot.

What I like about this domain is it’s art and science. There is a great deal of science behind our vision works, how humans see images, how we interpret them, what areas can you compress a little bit to kind of cheat based on how humans see things. You know, pull some of the reds out or blur the reds in the in an image and but still deliver an image faster. That’s how JPEG works. We’ll talk about that. You know, little things like that. But it’s also hard—well, how far is too far? “You know, this image is getting a little too soft. We’re cropping out someone important’s face”. It’s this ongoing, never-ending dialogue and tension between the visual designers in the tech world. And the programmers, people who are, you know, trying to get squeeze every last bit of performance out of their online experiences.

You know, I talked earlier about the conversation and relationship between the people responsible for creating the images, the designers, photographers, visual designers and the developers who are responsible with delivering those images. So that’s an ongoing dialogue that needs to happen. So both of those parties need to have a very good handle on image formats. I think the simple version of that conversation is if you have a photograph of a person of a scene, if it’s a continuous tone, photograph, JPEG is probably the easy choice. other formats like PNG, GIF, WebP, we’ll touch on SVG. It’s kind of like there’s photos, and then there’s everything else. And then there’s some caveats that we’re going to talk about that it’s important to learn if you are going to master images online.

Once you understand which format to choose, then it’s appropriate to learn a little more about how compression works, right? So this particularly will come in when you try to push that envelope, if you you know, don’t need to compress your images a whole lot. And your digital experiences, for example, are not super image heavy. You can kind of like set some basics. Does it look good? Does it load fast, you’re probably okay. But when you start pushing that envelope, and you start really trying to squeeze the most out of your images, that’s when it is important to understand. Now in general, and there are exceptions to every rule here. But in general, with JPEG, again, the continuous tone photographs, people places, objects, that’s where JPEGs fundamental approach to compression is lossy, that means you will if you keep pushing it, if you keep compressing it, it will start to visually break down you will see visual artifacts that will get blocky once you get into the very frontier of pushing it to the max you’ll start to see it separates the Luma that is the lightness In darkness of an image from the chroma, which is the color or human vision doesn’t notice when you compress the chroma, if you see the detail and the edges around the shape around a face and round someone’s eyes, those will be retained. Imagine a black white image. It’s almost like if you think of it as a colorized image, like an old black and white photo that’s been painted on, if you blur the colors a little bit like it’s okay, then that’s kind of how JPEG works. It’s actually a kind of an ancient by digital standards. It’s actually a fairly ancient algorithm. It’s not the most sophisticated that could be designed. And as we’ll see, when we talk about WebP, there’s some better algorithms out there.

PNG (Portable Network graphic) formats. A good use case for PNGs are things like infographics, logos, illustrations, diagrams, screenshots. shots. So the nice thing about paying as a format is that it’s fundamental compression algorithm is lossless. So that means that the more you can compress it, and you won’t see a lot of visual degradation, one tool that I use all the time, it’s called ImageOptim. It’s a little Mac app that I can just throw a batch of images on, and it will compress those for web, it will actually take PNG and do a slight amount of lossy compression. So you can that’s a setting you can do. You know, some of these formats are mature enough to where you can push them even further, if you know what you’re doing. Now, there’s some advanced techniques. And again, that’s where that art and science comes into play. It requires experience, it requires understanding the underlying technology a little bit.

WebP is definitely one to watch. If you are professional that is handling or delivering anything So WebP was created by Google back in 2010. It is a much more modern set of compression algorithms. So it provides both lossless and lossy compression. The biggest advantage is smaller image sizes without compromising on quality. It has a lossless mode, they can be 26% smaller than comparable PNG files, lossy images can be well, you know, if it’s like photograph continuous tone, they can be 25 to 35% smaller than comparable JPEG images. It also has an alpha channel. So it sounds fantastic, right? Like why wouldn’t you use it all the time? Well, the biggest problem is because it’s new. It has some browser compatibility challenges. So I’m looking at the CanIUse page if you go to CanIUse.com and look up WebP the currently the only big gaps are Internet Explorer. And that’s kind of old, that’s going to be sunsetted pretty soon. So that’s not a big problem. But Apple has really been a problem here with Safari on desktop and iOS is not supported. Now—the good news is the next versions of those browsers to come out Safari and Chrome for iOS 14 and Safari 14 Technology Preview currently support that. So I think that is going to change pretty soon. And we’re seeing people updating their browsers at a much more frequent rate now these days, so I’m thinking within the next year, this question will completely change.

SVG is a fundamentally different format. And it’s related to this conversation, because it’s a thing that designers have to touch. SVG is actually quite an old format. I think it was first created in the late 80s. It’s similar to HTML in structure. So you’ve got this onion skin layering of the code structure. It’s a document you can just edit, you can actually open up an SVG in a text editor and change stuff and play with it if you know what you’re doing kind of stands apart from these other formats in type, but it is something that designers and developers need to be educated about. Because you can do quite a lot with SVG, SVG support really kicked up, I don’t know, five, seven years ago. Now you can animate them, you can do all kinds of stuff. What Flash was in the early 2000s—and I’m an old Flash guy. Loved it. It’s some of the things that we’re only now getting around to be able to do today with SVG with image transparency like PNG we were doing in Flash 15-20 years ago, almost right. So SVG is really, really helpful because it’s math. It’s based on bezier curves. It’s based on vectors, right? So that’s different than all the other image formats because it’s vectors. That means that’s lines as opposed to pixels. It is something that can be scaled, resized, infinitely without any breakdown in the quality and also is highly, highly performant when it comes to file size. You’re loading your SVG by like doing crazy things by embedding other images inside of it or other stuff like that you can create really complex and very compact images through SVG. So I would encourage you to learn SVG. There’s a lot of great guides on the internet, @SaraSoueidan on Twitter is one of my favorite resources for that. She’s got some great guides that will help people get the most out of SVG.

Yeah, that’s right. It’s really useful. Now you need to have the tools to create them. My favorite tool is Figma. For that I’ve also used Sketch, Adobe Illustrator will do it. There’s some other tools like Inkscape think they’re still around, Affinity Designer, I think can create SVG. A lot of applications can do that now. And people in the design space are generally familiar with them, you’re starting to see it more and more because it’s beautiful and scalable. There’s still a few hangups with the format, I would say really the workflow, things like embedding typefaces, typically you have to outline them some design insider stuff that you might need to be aware of, if you’re working with them. If you’re a web designer, and you’re using tools like Figma, or Sketch, you’re going to be encountering SVG.

Now, there’s a couple sort of things to know if you’re a Contentful customer, but also this would apply to any CMS. I think, depending on what you want to do with the SVG in the digital experience. You have different options with how you can deliver these SVGs. So you can upload SVG use to Contentful via our assets screen and deliver them just like an image and then you can just access it like an image, you know, image source file path. And then [filename].svg, that’ll just display it is handled like an image just like this sort of blob and you lose a lot of the accessibility and flexibility benefits by doing it that way. That can work, and there’s situations where that makes sense. But you’re maybe not getting the most out of it. If you want to target the parts of the SVG, let’s say you’ve got an SVG of a car, and you want to show the wheels spinning or the car bouncing up and down. Like if you want to animate it with CSS or JavaScript, then you’ll want to actually inline that SVG into your front end experience, right? That means you’re going to inject it. So that means you’re probably not delivering that SVG with Contentful. You’re probably building it into your delivery stack. I was working on a Middleman project, I had a Middleman Ruby gem for embedding and inlining SVGs. What that does is, that makes the SVG part of the DOM (the Document Object Model), so then you can manipulate it, you can do cool stuff with it, it’s a lot more flexible. It just means it’s a little bit of different workflow. And the last point I’ll make and then that’s really probably not a best practice, but you can in a pinch, and that is the Contentful Images API can do a basic transformation of SVG into another format. Like, you can convert it from SVG to PNG or JPEG on the fly—but it won’t exceed the pixel dimensions of the SVG. So you can’t scale it up with that. Unfortunately, that would be super cool. And I might add that if people want to do that as a product suggestion, but it’s useful for fallbacks, if you’re dealing with formats, if you need to just spit out an icon works fine.

So these are some areas that I encountered in my journey as a photographer, right is dealing with these RAW image formats and every vendor—Nikon, Canon, Sony, they all have their different RAW formats. And the key distinction to understand is [that] it’s a mastering format. So just like Photoshop PSD files are what a designer or artist would use to composite and create all kinds of fantastic things, those are going to be very, very large. We don’t deliver those directly on the web. So RAW formats, like Adobe’s DNG format, there’s a couple others, Canon RAW, they are essentially inappropriate to deliver [to web]. Now, you’re going to want to run those through your system, like my favorite has been Lightroom. I’ve used Lightroom for many, many years. Typically, what I’ll do is I’ll export those out of Lightroom, either to my machine, or one of my favorite things is using a service like Smugmug, which has a direct integration. So if you’re a professional photographer, and you’re dealing with images, exporting them out of your raw editor, in this case, Lightroom via a plugin, something that I haven’t explored that would be interesting to find out is is there a way to make a Lightroom plugin that upload directly to Contentful. Because that would be super cool.

Responsive images allow you to deliver the right image, dimensions and pixels, the right size to the right device. And so one tactic is to use srcset and it will only load the one that’s appropriate to the viewport. Basically what you’re doing is instead of delivering one image, you’re delivering an array of images, you’re delivering a list of images. And it’s on us as designers and web developers to curate the sizes of that. Sometimes they might even be different shapes [aspect ratios], right? You might be delivering a square format on mobile, and you might need to go extra wide on my desktop. Let’s say you’re doing a headline banner image, right? That covers the top third of the screen or whatever. That opens up some interesting challenges when it comes to art direction. Let’s say you have a photo of people and they’re on the left side of your banner image, and then you’ve got some text on the right. So how do you crop that? Do you just let it scale down as you as your viewport shrinks? Well, then you can’t see the people you know, and then you can’t see the text. So you might actually need to create a separate image for smaller viewports. Or you might need to get smart about how you do the cropping. We have some face detection features that are very handy for cropping around the face, you can kind of set a focal point manually and once you get into more advanced images. With some of our App framework extensions, you can do things like AI image tagging, you can apply machine learning to images. And that will sort of work into the overall challenge of responsive images.

So I was a Contentful customer for about three years before I joined Contentful in this last January. And this was absolutely one of the capabilities that drew me to Contentful was that I could upload one master image that I had really gotten clean in Photoshop or whatever my design tool was. And then I could reuse that for all the different srcset dimensions for my responsive imagery because I wanted to deliver them and I wanted them to be fast, I want those high [Chrome Speedtest] Lighthouse scores. I want my SEO to be as good as it can be. I felt like mastering this as a digital craftsman was important to do. So definitely you can see some examples in the blog posts in question that I know you’ll share in your show notes where I break down like, here’s a photo of a lovely young lady. And we’re going to take that in crop that down into, say an avatar, basically, all you do is you tack on some attributes, things like width, height fit targeting the face, you can pick a region of the photo. So as a developer, when you’re creating the front end experience, you can just add those attributes on and it will just deliver that image. It’s basically creating a new one, you just set those attributes, it generates the images, our CDN delivers them really fast. The first time you hit it, it caches those. So that means that all your users that hit it at scale in the millions and hundreds of millions. They’re going to get those cached images. They’re very, very fast. It’s just nice to as a developer not have to worry about like “well, where are images stored? How do I handle that? Am I costing my company money by having to think about that?” You don’t have to think about it anymore.

We had a customer come through, they were selling travel stuff, boy, their images were just fantastic. Just big, gorgeous images, famous people, lifestyle stuff. They were fantastic. They had this problem, too. They’re like how do we make our user experience faster. So having that conversation, that ongoing dialogue between the people creating the images and the people delivering the images, that’s the creators and web developers, photographers, and the designers, engineers, front-end specialists, infrastructure, people who are responsible for delivering those images, you want to find that balance, and sometimes the balance is going to be more on the compression side. Other times, it’s all about that big, beautiful image: “hey, we’re gonna take your time, we need you to look at this gorgeous thing”. Sometimes those are fun as well.

Yeah, that’s a good point. Like I’m thinking of a guy, his name’s Jesse Brackenbury. He’s a landscape photographer. The guy is always up in the wilderness, crack of dawn, getting these incredible images in his sight is just stunning. You’re there for the images. That is what he is about. He is selling you prints and calendars and other things. And yeah, absolutely. Quality is paramount [in that scenario]. And people are probably willing to wait to see those incredible images.

Note: I finally was able to meet with Jesse and join him on this hike and shoot.

View this post on Instagram

A post shared by Jesse Brackenbury // PNW (@jesse.brackenbury)

It’s stored in our systems, essentially, it lives in the cloud. That’s several layers of abstraction. And actually kind of depends on where you are in Planet Earth, whether you’re getting delivered by Fastly or Akamai. So there’s some technical underpinnings there. The good news is, as a developer, it’s not something that you need to worry about—and I don’t say that to dismiss the concern at all. It’s just that it’s in practical terms, all you need to know is here’s the image, this API, here’s the endpoint for it. Here’s how I get my image. And then here’s how I deliver it. And that’s all in our documentation.

The Contentful Images API allows the retrieval and manipulation of image files referenced from our Asset store. So you can store all kinds of things in Contentful. We call it the Images API and not the Asset API, because images is the most common thing by far and 95.99% of what people use it for as opposed to delivering like files and PDFs and documents, etc. Okay, so it’s to let you, as a developer, retrieve, manipulate, and then deliver that image in an optimal way. Now, when you hit the API, if you’re developer, you’ll get a JSON response. Or you can represent it, think of it that way. The URL is the most important thing that you’ll always be getting. You can get other details like title, file name, content type as a JPEG, is it PNG, etc. Some details—it’s actually not a ton. So this is an area that I’m hoping our product team will be expanding on. We did just roll out a new feature called Tags, I believe they’re near-public, most of our larger customers are already using them. But basically, there’s now some native tags, and you can read about that at Contentful.com. But this is something that people who handle images and manage them are gonna care about, you know, adding tags, like is this featured image, is it outside, is it daylight, is it you know, what kind of thing is it? Is it a product, image, whatever. So now, those are some things that you can now get when you talk to our API’s. Basically, though, the job of the API is to let you deliver that image and you pick your parameters, you can add some parameters on top of that, in order to convert the format, compress it more, add some flags, like we haven’t really touched on things like progressive JPEG and eight bit PNGs. WebP is a whole other thing. You know, if you go to our developer documentation, you can learn all of the different things you could do with it. And the blog post covers, I think, the high points of that.

Let’s say you have a blog post and you’re trying to deliver that blog post, you’re querying the Contentful Delivery API (CDA), so it would see a referenced asset, if you go to our documentation, you’ll see an example of the fields you could get of what that would be like. But that would be within the overall API response of the larger piece of content. Yeah, you can still leverage the Images API, add those attributes on the end of your URL, etc. But they’re [the additional paramters are] not going to be in the response here. It’s up to you as a developer to add those.

Progressive JPEG is a fairly old sort of flavor of JPEG. What it does is it gives the browser successive passes at it. It’s pretty ancient. I’ve used it a couple times when I’m trying to push the envelope again, that’s where knowing this stuff really pays off, you know, your art director saying, we can’t shrink this anymore. This image, it’s got to be big and gorgeous. You know, I need to see Serena Williams lounging on this luggage. Yeah. And that’s an actual image, by the way, and it’s fantastic. And it leads to look amazing, but the developer is saying, or the engineer saying, No, we’ve got to be faster. That’s where you need to have this education to educate yourself to go and take that further progressive JPEG, I would say use it as an experiment. If you’re doing your responsive image delivery, well, then you’re probably not going to need it. A bit. PNG is another format. If you’re really trying to push PNGs are you’re using very small PNGs and you don’t have transparency. It can be very helpful there. There’s been a few times I’ve used that, not too often, and didn’t really touch on GIFs, but it’s definitely a format that people use. We don’t do any transformations on that I think you can resize it that’s about it probably is gonna look like garbage. If you do that, you know, GIF, it’s universal, works everywhere, but it’s kind of bloated. So if you’re doing a lot of animation, definitely look to video as something that’s going to be a little more efficient.

So that’s an interesting question. And that came up for one customer, I think it was in the automotive industry. So imagine use cars, you’re selling cars, you’re trying to buy them you’re trying to look at a lot of product photos—where do those car photos live? So is interesting in this industry, the industry standard is this one player that basically manages datasets of those and they’re external to Contentful but they’re all about storing the VIN of the car, the make the model they have that huge database, and they have product images, all that stuff that’s an external source. The question with the customer was do we try to put these in the Images API, or not? It was an interesting conundrum. Because that external source of data that, you know, they would sell to, you know, pretty much everyone in that industry, they didn’t have an Images API, you couldn’t scale them on the fly, you couldn’t do a lot with them. We see this in other areas, too, like e commerce, product providers, you know, you’ve got Commercetools, and Shopify and a couple others that are basically designed to work with headless commerce. Well, commerce, you know, a fundamental part of it is image handling. Some of those vendors have, okay, image API’s, and some of them don’t have any at all. And it’s entirely on you to deliver those. So it’s kind of like do you then just not have a good experience and rely on that external provider? Or do you try to master it and somehow synchronize it with Images API? So I will say the common use case for integrating a DAM (digital asset manager) like Cloudinary, well, I think would count as one, it’s more image specific. But then you’ve got the Bynders and Brandfolders are two, I think that we have app integrations with the cool thing with those is, let’s say you have a very busy agency model or a customer that just has tons of images, you have a lot of let’s say, You’re cranking out all kinds of things upstream outside of your web experience, brochures, you know, subway posters, videos, all this other stuff that doesn’t need to live in Contentful. But you have heavy workflows around those upstream of Contentful. Can you bring those things into Contentful, synchronize them and then leverage the Images API. It’s possible, but I don’t see that many people doing it. It’s complicated. Yeah.

Contentful’s core value proposition—this is really the world that developers are pushing us towards. And that is headless [content] customer experiences. We position ourselves as the content platform for your entire company to deliver customer experiences. Others focus on things like inventory management, customer data, embedding analytics, into the whole thing, the new skill is integrating all of that. So we have some direct integrations with some of those things and others that that wouldn’t get directly integrated; the integration point is going to happen in the customer application.

So I want to talk about two areas. One is what our customers at Contentful are asking for, and then another other things that I want, I’ll make a distinction there. So our customers aren’t pushing us terribly hard on this front. And I think part of the reason for that is the Images API, we’re doing the basics fine, in my opinion.

Now what I want as a creative person is I want other transformations, I want to be able to do duotone images, I want to be able to blur the images and add filters, I want to D saturate them, I might want to invert them in some cases. And I can think of use cases for all of those things. Another area that we only touched on briefly, and I won’t go into too deep because it’s such a deep area. And that is lazy loading, basically loading a very simple preview image in order to just to unblock the page loading. If you look at the way pages render images block that so the very first pass like let’s say you have a ton of images on your view or your page, you want to load some very low quality previews. You don’t want to look like garbage either, right.

So one technique I’ve seen is to actually generate an SVG that’s like an outline of the image. That’s so cool. You couldn’t do that with Contentful Images API. Today, you would have to do that on your application side. Another technique that I use on my website, if you go to allanwhite.design is if you look into the work section of the case studies, I blur the image [on load]. Now, that particular site, I designed it before I joined Contentful ;), it’s integrating with a company called Imgix. And they have a whole raft of image processing things that you can do. Now, our application integration with Cloudinary also lets you do the same things. As a developer, I found Imgix’s a little easier to navigate, so I went with that, but Cloudinary will do a lot of that stuff, too. So that’s where if you need to go beyond the normal if you need to go beyond what we have out of the box. We’ve got integrations with organizations like Cloudinary. It gives you other things too. It’s not just image processing, its image metadata. It’s advanced, you know, there’s metadata embedded in JPEG images like IPTC, geolocation, it could be tags. It could be a your camera information, you know, what camera was it shot on? All that stuff is embedded in images. And if you need that kind of thing, you want to have an integration with something like Cloudinary.

You’re very welcome. And I’m glad to be here; thank you for interviewing me. And I appreciate you reading the blog post. And I’d love to hear other questions that we can talk about in the future. Twitter.com/allanwhite is definitely where I hang out. And you can also find me in the Contentful community slack if you go to Contentful.com/slack, I definitely recommend it as a place to go just to navigate your way to get started, have questions answered. You will find me in the community slack answering questions sometimes.