Note on Crime – there have been some issues with the source data requiring additional processing checks. We aim to bring this dataset up to date in December.
September
Datasets that received updates:
Universal Credit (August + July update)
Claimant Count (August + July update)
Waste Data ( minor fix)
Dwellings data ( might just have been a fix too)
October / November
We have experienced some issues with changes to geographies used in the service that have led to data not passing our quality assurance checks. We expect to get the updates back on track through December and January.
Population Estimates
We are aware that the mid-2022 national and subnational population estimates for England were published last month. Our target is to add these to the National Data service by the end of December.
When the Filter button on the Data Explorer home page is clicked, the filter modal window now opens instantly. Prior to this update, this window could take 10 seconds or more to appear.
Decimal places
Up until this update, the Data Explorer had no capability to enforce a consistent number of decimal places for numerical data values. In other words, the application would simply display the values stored in the feature service in ArcGIS. An example is shown below – in the 2013 columns there are values displayed with 0, 1 and 2 decimal places (this is how the values appear in the underlying feature service).
From this update, the Data Explorer will now look for a field with a name of DecimalPlaces in the metadata table for the data catalog. If this is found, the Data Explorer will use the values in this field to set the display decimal places for each indicator. In the example above, if the decimal places field contained a 2 for the Persons age group 0-4% indicator, all data values in the table would be displayed to two decimal places. If there is no decimal places field in the metadata table, the Data Explorer will revert to displaying the values as they are defined in the underlying feature service.
Metadata ordering
Control over the display of metadata in the Data Explorer has been improved. Using the Metadata Fields setting in the application configuration, the owner can control the list of fields that will be displayed in the Data Explorer, irrespective of whether the fields in the metadata table were supplied as standard, or were subsequently added as custom fields. There is now also complete control over the order in which metadata fields are displayed, simply by changing the order in the configuration setting. Again, it is possible to reorder elements irrespective of whether they are standard or custom fields.
Data export
The data export has been improved so that it now contains custom as well as standard metadata fields. Also, when the Data Explorer is running in the German locale, the CSV export format now opens correctly in Excel (; as delimiter).
German text
Minor changes have been made to the interface text when running the Data Explorer in German.
We hope you will enjoy the new version of Data Explorer!
Crime data (May 2023) was updated on 14th of July.
Economy & Employment
Claimant Count for May (updated) and June (new) was updated on 11th of July.
Universal Credit for May (updated) and June (new) was updated on 11th of July. Except for the employment status based indicators which are available until May (new) as these are released in arrears.
Business Demographics (2016 to 2021) was updated on 24th of July.
Health
Public Health Outcomes Framework (as at 19th of June) was updated on 24th of July.
Housing
House Prices (Jan – Dec 2022) was updated on 24th of July.
Crime data (Mar 2023) was updated on 16th of May. A further update was then made the following week to resolve a numerator/denominator issue that was causing problems in reports with data aggregations used in the Custom Area Reporter.
Economy & Employment
Claimant Count (April 2023) was updated on 18th of May.
Universal Credit (April 2023) was updated on 18th of May.
Health
The Wellbeing dataset (2021-2022) was updated on 22nd of May.
Crime data (Feb 2023) was updated on 5th of April.
Economy & Employment
Claimant Count (March 2023) was updated on 22nd of April.
Universal Credit (March 2023) was updated on 24th of April.
Annual Population Survey (Jan-Dec 2022) was updated on 24th of April (still no occupation or industry data update, annual qualification data not included).
A correction to the NEET data was applied on 24th of April.
Environment
Waste Management data was updated on 26th of April.
Health
PHOF and CMH data from Fingertips as at 31 March was updated on 20th of April with some fixes applied on 24th of April.
The ward geography in the National Data Service has been updated to 2022 wards and we have started progressively rolling this change out to customers’ data catalogs. We expect this process to be completed by the end of this month.
For InstantAtlas Data Observatory customers, this means that it will be possible to view Census 2021 data in ward-level standard outputs within the observatory website, wherever data was published at ward level (not all of the phase 1 datasets were published by ONS at this level).
The process of updating the designs of ward-level reports to replace Census 2011 data with Census 2021 data has begun, and will be ongoing until the end of the month.
This includes the ward-level reports used in the Custom Area Reporter app to show aggregated data for a custom selection of wards.
The data from the UK armed forces veterans topic was published on 14th of January. Please read this post for more information.
Economy & Employment
People in Universal Credit (September 2022) was updated on 13th of January.
Claimant Count (September 2022) was updated on 30th of January.
Annual Population Survey, ASHE, Business Counts, Job Density were updated on 30th of January.
Note that NOMIS has since issued a warning that there were errors in the published APS data, which have now been rectified. As a consequence, we will have to re-run the update in order to correct the figures.
The speed of non-census updates has been impacted by the ongoing processing and loading of new Census 2021 data. However, we expect these updates to return to normal as the census data processing reduces from February onwards.
Ethnic group, national identity, language and religion
Labour market and travel to work
Housing
Sexual orientation and gender identity
Education
Health, disability and unpaid care
To give an idea of the scale of the undertaking, for Census 2021 we have now added about 3,800 indicators, 70 census tables and over 100,000 instances to the National Data Service.
These new indictors are located under different themes depending on the topic summary. The table below shows the top level theme in the National Data Service in each case.
Census 2021 topic summary
NDS top level theme
Sub-themes
Ethnic group, national identity, language and religion
Population
Ethnicity, Identity, Language and Religion
Labour market and travel to work
Economy
Economic Activity Status; Travel to Work; Industry and Occupation
Housing
Housing
Households; People with second address; Communal establishments
Sexual orientation and gender identity
Population
Sexual orientation; Gender identity
Education
Children and Young People
Qualifications; Population and Pupil Characteristics
Health, disability and unpaid care
Health
Disability; General Health; Unpaid Care
As with the previous updates, we have retained the structure the data were published in, and have also retained the name of the census table in the theme names and indicator names. This is helpful for users that are familiar with the way the census data is organised and/or the names of the tables.
Under each top level theme in the National Data Service, the old data from Census 2011 have been moved into a subtheme labeled Census 2011, to avoid confusion with the 2021 data.
Note that it is not the case that all data is available at all geographic levels. For example, the ‘detailed’ tables for Ethnicity, Identity and Religion do not contain data at LSOA 2021, nor do the tables for Sexual Orientation and Gender Identity.
Data is generally not currently available for the Ward geography. We are in the process of updating the ward definition in the National Data Service to the latest (2022) wards and as part of that process, Census 2021 indicator will gain data that level wherever it has been published by ONS.
We will also be adding descriptive metadata for the Census 2021 indicators held in the service.
For customers with a managed Data Observatory website, the data are displayed in the following places:
1) Map Explorers (updated 3rd Feb 2023)
2) Data Explorer (updated 3rd Feb 2023)
3) Reports (updated 3rd– 14th Feb 2023)
The reports for upper and lower tier local authorities have been updated to replace 2011 census data with 2021 data. New reports for the LSOA 2021 geography have been added for the Population, Health and Environment themes (the other themes retain reports at the old LSOA geography for now as there is more relevant data available at this level for the time being).
Reports for custom geographies will show Census 2021 data wherever we have been able to aggregate data to these geographies from 2021 OA or LSOA levels. If any of your reports are still showing data for 2011, it may be that you need to send us an updated lookup for your custom geography in order for us to apply the aggregations.
Note that LSOA 2021 reports have also been made available in the Custom Area Reporter app (for relevant themes), if your observatory website is using this app.
