Taking Responsive Images to the Next Level with Focused Responsive Images

Responsive web design has become a basic asset in every web developer’s life. One big topic in responsive web design that is still only partially solved and not yet proposed to the html5 standard is responsive images. This Article will show some concepts that already exist but also a new concept that was developed by Cambridge Technology Partners in order to simplify the information management process with responsive images.

responsive_devices

Background

The idea to display content differently on different devices is quite old and was already proposed to the CSS1 standard back in 1994. We also all know the media-dependent cascades introduced in HTML4 that are famous for their uses for print styles:

<link rel=“stylesheet” type=“text/css” href=“print.css” media=“print” />

The proposal for HTML4 in 1997 was even going a bit further than that and described possible uses in the future. The following quote (http://www.w3.org/TR/html4/types.html#h-6.13) taken from the original proposal of HTML4 shows that they already thought about parameterized media attributes in the stylesheet links and @media conditions:

Future versions of HTML may introduce new values and may allow parameterized values. To facilitate the introduction of these extensions, conforming user agents must be able to parse the media attribute value as follow:
The value is a comma-separated list of entries. For example,
media=”screen, 3d-glasses, print and resolution > 90dpi”

The main development of the media queries as we know them today started already in 2001 and was initiated by Opera developer Håkon Wium Lie (http://www.w3.org/TR/2001/WD-css3-mediaqueries-20010404/). It took 11 years until media queries became a W3C recommended standard in 2012 after further development by Florian Rivoal. Together with the emerging mobile device trend breaking the global internet usage statistic of 1% (http://www.w3schools.com/browsers/browsers_mobile.asp), CSS3 media queries laid the groundwork for responsive web design as we know it today. Now it became possible to build queries on media type attributes and use them to enforce different styles based on logical rules.

The same queries can be used embedded within a style sheet for conditional sections and is probably the most common usage today:

Responsive Images Using CSS

With the possibilities to use different layouts on various devices that come with a high range of orientation dependent viewport sizes it became clear that images also need to adapt to this variety. Pictures that look great on multi column 16:9 desktop layouts need to change their look if displayed on a small screen 320px display. This could simply be achieved by using media queries and resizing the dimensions based on conditions. The following example shows a mobile first approach where screens smaller than 320px in width will show images in an article content with 160px in width and screens with a higher resolution will show the pictures with 320px width:

Still we face the issue that on different layouts prepared for different media dimensions we will not only want to display images scaled differently but also display a different crop in other dimensions. A big 4:3 content image may get a small horizontal stripe on mobiles.
Showing the same image source with different dimensions on different devices brings a whole lot of other issues. The simplest solution would be to specify the exact dimensions to use for the image on each media. This however will destroy our original image proportion and will result in a distorted image. To get around this issue we will need to crop the original image so it can fit into our new dimension and use a cropped version of our original image on each media. We could still achieve this purely in CSS if we are able to use background images for the given situation. A responsive image solution with CSS using background images could look something like this:

As the situations where we actually are allowed to simply use a background image are quite limited due to accessibility / SEO constraints and for simply not violating the semantic correctness of an <img> tag on content relevant images , we can rarely stick to this solution.

Responsive Images in HTML

As outlined in the previous section there is still a need for HTML image tags to be able to respond to media changes and change image sources based on viewport changes. Unfortunately the HTML5 and CSS3 standards did not include any concepts to address this problem. However, there is a W3C community that started to develop concepts for a proposal to include into the W3C standard. The responsive image community group emerged out of a great discussion around the need for responsive images in HTML as a standard that started on whatwg in February 2012 (http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-February/thread.html#34634). There were two initial proposals where one of them is now on the way for final specification request to be picked up by the W3C standard. In the future it will be possible to use a new <picture> tag that will allow specifying multiple sources for different media (http://picture.responsiveimages.org/). Using the picture element would look like this:

This concept includes a fallback image for old browsers (specified by the img element inside of the picture element) and different sources for the different media types. In this example there is a small image used but overridden by the sources with the min-width rules if applicable (mobile first approach). The picture element is still in in W3C draft mode (http://www.w3.org/TR/html-picture-element/) and it will also still need some time until the browsers will adopt the specification and provide an implementation. So what can you do in the meantime if you still want to use responsive images?

Picture Element polyfill

While the element is still in draft specification it’s pretty clear already that the proposal will be more or less what it is right now (some small changes on the srcset DPR switching options for hight DPI devices are still in discussion) and the direction with the picture element is what will be included in the W3C standards. If you would like to use responsive images already and come close to the standard how they most likely will be used in the future it’s quite a good choice if you’re using one of the existing polyfills that are out there. The most popular one right now is probably from Scott Jehl and it already implements the specification outlined in the W3C draft for the picture element. Scott uses span elements to not break old browsers. The polyfill is then relying on the JavaScript matchMedia function to determine matching media rules and switching the pictures based on that.

Developing Your Own Responsive Image

While the picture element specification and the polyfill from Scott Jehl provide already a comprehensive solution for the responsive image problem you still face some drawbacks with this solution. The main issue with the current concept is that the breakpoints need to be specified over and over on each picture element you have on your site. It would make much more sense to centralize the breakpoint settings and use them with sort of aliases. This was also outlined in the W3C community several times and the anti-DRY nature of the concept was questioned. In a recent project we have worked on for a client we have also realized this lack in the specification and saw other benefits of centralizing and aliasing the breakpoint settings. As you also develop javascript modules and components along your website they may also change their behaviour based on the media query breakpoints. With a centralized solution to manage your media breakpoints it would also be possible to make your components consume changes in these breakpoint settings and react appropriate. With the help of the matchMedia function in javascript and an event architecture like provided by jQuery you can easily build your own media manager where configured media breakpoints changes get broadcasted as events. Components and modules, including your own responsive image component, can then react on those events and change their behaviours. In our project we have developed such a framework based on media aliases and various components including our own responsive image component consume this service to adapt to media changes.

This example shows the initialization of a centralized media manager that is maintaining the media query aliases and broadcasts events on media changes. Components can then register to be notified on media changes and can react on them using the alias names. A sample usage of this would be a component hiding all images except if displayed on large sized screen:

We have also built our own responsive image component that is using the html figure tag to build the markup. We are then using separate data attributes that contain the alias names in order to specify the different images for the media breakpoint aliases specified in our responsive manager.

With this approach you can then dynamically find the right image using the media alias (s, m and l) in the data attribute and change the displayed image on media match events.

Hints and Pitfalls for Your Own Responsive Image Solution

In the project for our customer we also store an order number with the responsive aliases and a group name for the aliases. This way we were able to have a fall back strategy if one or more media breakpoint aliases are not configured for a specific component. I.e. if we take our responsive image component like shown in the HTML5 code snippet above and if we would remove the data-media-m attribute we would only have an image specified for the “s” and “l” media breakpoints. The order number we could specify in our Responsive manager for each breakpoint alias together with a group name would help to determine the next approximate image to display if the “m” breakpoint gets triggered. In our case we would look first for next higher available image and if nothing specified the next lower. This way we can only specify the required images with the data-media-x attributes and skip some breakpoints while not losing the image.
In some cases you would want to also not load any image on certain breakpoints. To handle this requirement we have included an option on our responsive image to specify optional excluded aliases directly on the concerned image. This would look something like this:

In this example there would not be any images displayed for the media aliases “s” and “m”. Together with the fallback strategy we have now a very comprehensive and flexible responsive image solution.
Be always sure that you don’t load images you don’t need to load immediately. The big benefit of responsive images is also that you can limit the image size footprint on small media by just loading the images appropriate for the given media size.

Server side image processing to automate responsive image creation

Currently responsive images only address the technical issue but not the content issue that comes with the big variety of different devices. The classic approach is that the visual designer provides all the different images for the media breakpoints. This could easily mean that for each image on your site the designer would need to prepare at least 3 versions of each image with the appropriate dimensions. If we are talking about a content driven web site this could easily mean the preparation and maintenance of thousands of images in their different versions.
In an ideal world we would have two roles managing and maintaining responsive images. The designer only cares about the dimensions of the images for each breakpoint while the content editor cares about the information that is displayed on the image.
Since we are lucky and have the whole processing power of the backend servers we could think of a solution for this problem here. One very good example of how to use server side processing to get different versions of the same source image can be demonstrated if we look at the placehold.it service for placeholder images. In our previous responsive image component HTML snipped you can already see that we have specified the image dimensions in the URL that we call at the pracehold.it service. The backend processing of the server will read those dimensions specified in the URL and provide an image with exactly the requested dimensions.
Applied to our role segregation of a designer and a content editor we could then say that the designer is specifying what image dimensions need to be used for the breakpoints and the content editor is only providing a high quality source image that is then processed by the backend server depending on the requested dimensions by the responsive image component.
The server side would then have different options to process the requested image with the desired dimensions. The most common two approaches would be scaling and cropping. With the scaling approach we would still have the problem that the image will be distorted when requested in a different proportion that the original image. Cropping usually happens as a so called center crop. This would solve our problem with the image distortion but we would just crop around the center of our original image and most likely lose very important information in our image.

Focused Responsive Image Concept and Focus Cropping

Cambridge Technology Partners developed a concept and provided a solution for something we like to call focus responsive images. As outlined in the previous section we would ideally want a solution where we can automate the whole image generation process for the different responsive images while not violating the design with distorted images but also not lose any content information by cropping important information from the image.
Focused responsive images are a concept that involves both server side implementation as well as client side components. The main concern we wanted to address is that content editor only need to specify one high resolution original source image and that an automated process will generate versions of those images according to the required dimensions that are defined by the designer and requested by the client. In order to minimize the risk of losing important information in the source image when generating cropped versions of the image we have developed an image processing algorithm that is providing what we call a focus crop of the image.
The content editor only needs to upload a high resolution version of the image and he then also specifies a rectangle area (we call that focus area) on the image where he thinks is the most important information located on the image. Of course there are images where the important information is distributed across the whole image but in most of the cases there is kind of a hot spot on the photo where the real message lies. The content editor is very well aware of this as it’s his responsibility to bring the information to the user.
The focus responsive image algorithm is cropping the image while taking the focus area into account. The algorithm is using cropping and scaling to find the ideal combination while minimizing the risk to lose important information and violating the focus area. A very high level description of the algorithm would be that it’s cropping as far as possible to the center of the focus are but starts to scale the image once further cropping is not possible due to the constraints of the focus area.
The following illustration shows you how this would look like on a website where three different image dimensions are requested from the same original image. On the left you see the original image and requested image dimensions with dotted lines. The red dotted line shows the focus area that was set by the content editor on the original image. On the right side you can see how the algorithm behaves on different requested dimensions. It always tries to crop around the center and starts to scale the image if the focus area gets violated.

responsive_concept

Summary

Responsive images are a key asset in today’s web design and will soon be integrated into the W3C standards. The main issues right now come with the additional resources that need to be produced and maintained while server side processing can help us to control or even eliminate the administrative overhead behind the maintenance of responsive images. The focus responsive image concept and implementation that was developed by Cambridge Technology Partners shows that there is a comprehensive method to modify content images and provide content for different media automatically.

Summary
Article Name
Taking Responsive Images to the Next Level with Focused Responsive Images
Description
Responsive images are a key asset in today’s web design. There is a comprehensive method to modify content images.
Author
Publisher Name
Atos Consulting CH
Publisher Logo

Leave a Reply

Your email address will not be published. Required fields are marked *