Designing a City Rent Prediction System
You are designing an end-to-end machine learning system to predict rental prices across a city, leveraging structured listing data, text descriptions, images, and geospatial features.
Address the following:
(a) Data collection and cleaning -- What data sources would you use? What cleaning steps matter most?
(b) Feature engineering -- How would you extract useful signals from text descriptions, listing photos, and geospatial data?
(c) Model selection -- What model architectures would you consider, and how would you combine structured and unstructured features?
(d) Validation -- How should you split data to avoid leakage, given that listings have both temporal and geographic structure?
(e) Fairness and robustness -- What biases are you worried about, and how do you detect and mitigate them?
Hints
- Start by identifying the three main leakage risks before designing anything: temporal leakage (training on future data), spatial leakage (testing near training neighborhoods), and feature leakage (using variables only available after prediction time).
- For combining structured and unstructured features, a common practical approach is to extract embeddings from text and images separately, then treat those embeddings as additional numeric features for a gradient-boosted tree -- no end-to-end neural training required.
- For validation, the key design question is: what does deployment look like? If you will be predicting rents in new neighborhoods next month, your validation set should be new neighborhoods from the next month -- not a random sample of all your data.
Worked Solution
How to Think About It: Rent prediction is a hedonic pricing problem -- you are trying to decompose rental prices into contributions from a property's observable attributes. The core challenge is that you have multiple modalities (text, images, numbers, location) that need to be combined, and several types of leakage to guard against: temporal leakage (training on future data), spatial leakage (testing on neighborhoods geographically adjacent to training data), and feature leakage (embedding information that is only available at listing time, not at prediction time). A senior ML practitioner thinks about these failure modes before thinking about model architecture.
Key Insight: The biggest practical risk is leakage -- a model that looks great in validation but fails in deployment because it learned a spurious signal. Every design decision should be evaluated with the question: would this feature be available at the time of prediction in production?
The Method:
(a) Data Collection and Cleaning.
Primary sources: - Rental listings (Zillow, Apartments.com, Craigslist scrapes): price, sqft, bedrooms, bathrooms, amenities, listing text, photos, listing date. - Public records: property tax assessments, deed transfers, building permits, zoning maps. - Geospatial: OpenStreetMap (transit stops, parks, schools, restaurants), walk scores, crime statistics by census tract. - Temporal: days on market, relisting history.
Cleaning priorities: - Deduplicate relisted properties (same unit listed multiple times at different prices). - Cap outliers in price and sqft at reasonable percentiles -- do not remove them silently. - Handle missing photos or descriptions with explicit missingness indicators (do not impute with zeros). - Normalize price to dollars per square foot as an alternative target.
(b) Feature Engineering.
Numeric/structured: sqft, bedrooms, bathrooms, floor number, building age, parking included (binary), pet policy.
Geospatial: - Distance to nearest subway station, bus stop, highway. - Walk score, bike score, transit score. - Neighborhood-level aggregates: median household income, school ratings, crime rate -- all from census tract or zip code, lagged by at least 1 year to avoid leakage. - Spatial lag features: average rent of nearby listings in the past 6 months.
Text features (listing descriptions): - Bag-of-words or TF-IDF as a fast baseline. - Fine-tuned transformer embeddings (e.g., a lightweight BERT variant) for richer semantic features. - Explicit keyword extraction: doorman, laundry, renovated, exposed brick, dishwasher. These amenity indicators are often more useful than dense embeddings for tabular models. - Sentiment score as a meta-feature.
Image features (listing photos): - CNN-extracted embeddings from a pretrained model (ResNet, EfficientNet). - Structured attributes from a fine-tuned classifier: natural light score, finish quality (low/medium/high), presence of outdoor space. - Be careful: photo quality and count are correlated with price and listing quality, so include them as explicit features.
(c) Model Selection.
- Hedonic regression (interpretable baseline): $\log(\text{price}) \sim \beta^\top x$ with regularization. Use this to understand feature importance and establish a benchmark. Interpretable coefficients are valuable for stakeholder communication.
- Gradient Boosted Trees (XGBoost/LightGBM): Best for structured/tabular features with nonlinear interactions. Fast, robust to outliers, handles missing values natively. This is typically the production workhorse.
- Neural network fusion model: For incorporating text and image embeddings, use a multi-input architecture: embed text via transformer, embed images via CNN, concatenate with structured features, pass through a small MLP. This captures cross-modal interactions.
- Ensemble: Stack GBT and neural network predictions with a linear meta-learner.
(d) Validation Design.
Do not use random K-fold CV -- it leaks future information and spatial autocorrelation.
- Temporal split: Train on listings before date $T$, validate on listings in $[T, T + \Delta]$, test on listings after $T + \Delta$. This mirrors production deployment.
- Purging: Remove listings from the validation set that were active during the training period (to avoid using stale information from the same listing).
- Spatial cross-validation: Hold out entire neighborhoods or census tracts for the validation set, not individual listings. This tests whether the model generalizes to new areas, not just to new listings in known areas.
- Leakage audit: For every feature, ask: is this derived from data that postdates the listing? Common culprits -- using the final closed price as a feature, using aggregate statistics computed on the full dataset before splitting.
(e) Fairness and Robustness.
- Protected attribute proxies: Geospatial features (neighborhood, zip code) can encode race/ethnicity historically. Monitor model predictions for disparate impact across demographic groups using census data as a proxy.
- Disparate impact testing: Check whether the model systematically over- or under-predicts prices in majority-minority neighborhoods. Use demographic parity or equalized odds metrics.
- Distribution shift: Rental markets can shift rapidly (COVID, interest rate changes). Monitor feature distributions and prediction errors over time. Retrain on rolling windows rather than full historical data.
- Outlier robustness: Luxury listings and social housing are in the same dataset but follow different pricing mechanisms. Consider separate models or explicit regime indicators.
Answer: A robust city rent prediction system requires: multi-source data collection with careful deduplication; structured + text + image features with explicit leakage checks; GBT as the primary model with optional neural fusion for unstructured features; time-based and geographically-stratified validation splits; and ongoing monitoring for disparate impact and distribution shift.
Intuition
Hedonic pricing models have been used in real estate for decades. The modern version of this problem is really a lesson in how to correctly combine multiple data modalities and avoid the many ways a model can appear to work in development but fail in production. The leakage risks are subtle: spatial autocorrelation means that random splits overestimate generalization, temporal leakage from aggregate features (like neighborhood medians computed on the whole dataset) inflates held-out performance, and photo quality is a confounder that can encode landlord sophistication rather than property quality.
The fairness issue is important and practical. Geospatial features are powerful predictors of rent, but they also absorb decades of discriminatory housing policy. A model that predicts rent accurately using neighborhood features is not necessarily fair -- it may perpetuate existing pricing disparities. This is not a reason to remove geospatial features (they are too predictive), but it is a reason to audit model outputs across demographic groups and to be explicit with stakeholders about what the model is and is not doing.