Pixels are the basis of all digital displays and thereby, are what derive all visual displays we see when we interact with our electronic devices. They are also the metric we use to track how we interact with those same displays. The way a cursor is moved across the screen and the location of tapping a finger on the screen are both tracked by their pixel locations. Seems straightforward enough. However, all pixels aren’t the same and the locations they represent can vary. Especially on web browsers.

When Pixels Aren’t Quite Pixels

By definition, a pixel is “the smallest element of an image that can be individually processed in a video display system.” However, in practice this can create problems when interacting with displays of varying display densities. When talking about computer displays, a standard monitor is considered by most platforms to have 96 DPI (dots per inch). This means that in a given inch you should be able to count 96 individual pixels. Screens have gotten much better over the years which has created a need to adjust displays based on their DPI to ensure content is displayed in a consistent manner. For example, a 200 x 200 pixel image will look to be twice the size on a display with a resolution of 1920 x 1080 (1080 display) as it would on a display with a resolution of 3840 x 1260 (4k display).

Enter DevicePixelRatio

To address the issue of content being displayed at different sizes across different resolutions, web browsers started tracking the DPI as a DevicePixelRatio. The DevicePixelRatio is used to adjust the display within the browser window to keep the size of content relatively consistent between devices. This means a 200 x 200 pixel image will actually take up 400 x 400 pixels on a display with a DevicePixelRatio of 2.0.

Fuzzy Pixels

The usage of DPI makes for a much more consistent display experience but you have to be careful when reading pixel values to know whether you’re using DPI adjusted pixels or raw pixel values. In general the browser will give you the DPI adjusted pixel value when it gives you a measure of “pixels.” However, the DPI value isn’t always correct and sometimes you want to know the physical size of the display instead of the DPI adjusted logical size. For example, a 2019 16” MacBook Pro has a physical display resolution of 3072 × 1920 pixels. Inside a browser, we can get the logical size of the display by checking the Screen.width and Screen.height, which is reported as 1792 x 1120 pixels on Chrome. This can then be converted by multiplying the logical resolution by the DevicePixelRatio which is reported as 2.0. However, this gives a resolution of 3584 x 2240, which is 16.7% larger than the actual resolution of the display. For this reason it’s best to try to operate within the logical space as the DevicePixelRatio can be inaccurate or rounded within the browser.

Pixel Relativity and Viewport Sizes

Besides making sure things are displayed consistently between displays, another major use we have for pixels is tracking interactions within the screen. The Moonsense SDK uses pixels to report the location of all Touch and Pointer Events. The Moonsense SDK automatically records Touch Events within the logical space across all devices to make touch points DPI independent and directly comparable across devices. 

The location of these recorded events are important but it’s also important to know the coordinate system of these events and the relative location of that coordinate system. Within the browser there are many ways the viewport is sliced and diced. To put it in perspective, just trying to get the overall page size can be an adventure on its own. For example, just looking at width, the following properties can exist on the document.documentElement and document.body: clientWidth, scrollWidth, and offsetWidth. These are in addition to the Screen.width mentioned above. All of these have different meanings with respect to what portions of the browser are included in the measurements and how they are calculated. They are affected by things like margins, paddings, borders, browser window features, etc… All of these are important pieces of information that are taken into account when working with ScrollEvents within the Moonsense SDK.
When looking at where a touchpoint happens within a screen it’s important to note that the coordinates reported in the browser MouseEvent are split into Client and Screen coordinate systems. Client is with respect to the browser window and Screen is with respect to the entire display area, including other displays. Below is how the MouseEvent screenX and screenY locations appear on a multi-display setup. These are not adjusted based on the DevicePixelRatio.

For the purposes of fraud detection and use within the browser, these values are not as useful as knowing the location of the pointer with respect to the confines of the browser window. In order to track that value, we can use the clientX and clientY values of the MouseEvent. These values are displayed with the DevicePixelRatio factored in. It is very important to track which value you are interested in because they can look very similar depending on the device being used and the location of the browser window. For example, you can see below that the coordinates for the mouse click are only 100, 100 even though the click is happening towards the middle of the display.

Ok Great, But Who Cares?

Browser page and form interaction can be used as an indicator in the detection of fraudulent users. For example, you can see the article MOUSE Movement modeling to predict online Fraud by Pranay Dave (Director at Teradata), which describes using features derived from mouse events to create a model that can determine fraudulent activity. Combining this information, along with other sensor data, can greatly increase fraud detection or bot detection models. However, capturing and consolidating the data can be tedious, if not confusing and complicated. Especially when trying to correlate data captured across multiple devices with different methods for reporting the information. Fortunately, the Moonsense SDK can make capturing and handling the data much easier with our Touch Events.