Predicting housing prices in Hong Kong based on AI interpreted sentiment in social media, health and sustainability factors: A white-box AutoML research agenda

Abstract

In recent years, automation via deep learning/machine learning has flourished. As such, automated valuation will likely become a major task in valuation over the coming years. While most studies in industry/academia utilise multivariate regression for housing price predictions, we aim to introduce explainable AutoML models (H2O AutoML and Google Cloud Platform (GCP AutoML Tables) for housing price predictions, which will be the first of its kind. The objectives are as three folds: 1. predict housing prices based on the following factors: (i) sentiment, as reflected in (ia) animal spirits (market sentiment) interpreted and constructed using social listening tools with artificial intelligence, and (ib) a sentiment index constructed based on Google searches; (ii) health-related factors, including COVID-19 and radio base stations; (iii) sustainability factors, i.e., the urban heat island effect and polychlorinated dibenzo-p-dioxins (PCDD); (iv) the number of years left on a land lease; and (v) the impact of precarious social events on housing prices. 2. Study the impact of these factors and compare their relative importance on housing price predictions. 3. Predict housing prices via (i) H2O Auto ML and (ii) GCP AutoML Tables and compare their results. The first stage includes a systematic literature review and content analysis of big data housing price research for the past ten years in Science Direct, Emerald, Sage and Taylor, and Francis, to uncover the current state-of-the-art research in housing prices and revise/update the variables to be included in our prediction model. Fifty-five highest housing price transaction records in Hong Kong will be used for the housing price predictions. Housing estate name, price, date of transaction, gross floor area, building completion date, and floor level will be downloaded from the Economic Property Research Center (EPRC). We will then reformat the data so that it can be fitted to AutoML. Second, we will obtain the sentiment of 55 housing estates and the “Hong Kong’s housing market”, as reflected in social media, by using keywords to search on a Natural Language Processing platform. Third, PCDD will be obtained from the Environmental Protection Department, while radio base station locations will be obtained from the Office of the Communications Authority. The number of years left on the housing estates’ land lease will be obtained from the Land Registry Department, with dummy variables being assigned in the prediction models to represent the precarious protest period and the number of COVID-19 cases obtained from Statista to study their impact on housing prices. Moreover, the remote sensing data of housing estates obtained from Landsat 8 (an American Earth observation satellite) can be downloaded through the United States Geological Survey (USGS). The land surface temperature, computed by ENVI 5.3 software, will be compared with nearby areas of vegetation to compute the extra heat caused by the “urban heat island effect”. Big data collection will allow us to utilise H2O AutoML and GCP Cloud AutoML for housing price predictions. We will apply H2O AutoML to select the best machine learning algorithms. Unlike traditional machine learning algorithms, AutoML minimises parameter adjustments and automatically tunes all parameters according to embedded algorithms. The results will then be compared with the results from GCP Cloud AutoML. We will also select the model with the highest efficacy, which will reveal the relative importance of factors and show the coefficients of the factors on housing price predictions.