Transform Data Like a Pro: Mastering map(), apply(), and Lambda Functions in Python and Earth Engine
If you’re working in data processing or analysis, you’ve likely encountered .map(), .apply(), map(), and lambda functions in Python and JavaScript. These versatile tools allow you to efficiently apply operations to data structures, transforming raw data into actionable insights. In this article, we’ll explore their syntax, use cases, and specific examples relevant to water resources management.
1. Python’s Built-In map()
The built-in map() function in Python applies a function to each item in an iterable (like a list or tuple) and returns a map object, which can be converted to other iterable types like lists.
Syntax:
map(function, iterable)Applications in Water Resources
Example 1: Converting Units (e.g., Inches to Millimeters)
def inches_to_mm(value):
return value * 25.4
rainfall_inches = [2.0, 1.8, 3.2, 0.5]
rainfall_mm = list(map(inches_to_mm, rainfall_inches))
print(rainfall_mm) # Output: [50.8, 45.72, 81.28, 12.7]Example 2: Checking Flood Warning Thresholds
def flood_warning(flow_rate):
return flow_rate > 1000
flow_data = [850, 1200, 950, 1100]
warnings = list(map(flood_warning, flow_data))
print(warnings) # Output: [False, True, False, True]2. Pandas .map()
Pandas .map() is a method for transforming Series objects by applying a function or mapping dictionary.
Syntax:
Series.map(function/dict)Applications in Water Resources
Example 1: Mapping Land Use Codes to Descriptions
import pandas as pd
data = pd.Series([1, 2, 3, 1, 2])
land_use_mapping = {1: 'Residential', 2: 'Commercial', 3: 'Agricultural'}
mapped_land_use = data.map(land_use_mapping)
print(mapped_land_use)Example 2: Applying Functions to Analyze Water Quality
def classify_ph(ph):
if ph < 6.5:
return "Acidic"
elif ph > 8.5:
return "Alkaline"
else:
return "Neutral"
ph_values = pd.Series([7.0, 6.2, 8.7, 7.8])
ph_classification = ph_values.map(classify_ph)
print(ph_classification)3. Pandas .apply()
The .apply() method allows you to apply a function along an axis of a DataFrame (row-wise or column-wise).
Syntax:
dataframe.apply(function, axis=0/1)Applications in Water Resources
Example 1: Calculating Water Demand Per Capita
def water_demand(row):
return row['Total Demand (L)'] / row['Population']
water_data = pd.DataFrame({
'Region': ['A', 'B', 'C'],
'Total Demand (L)': [100000, 150000, 120000],
'Population': [5000, 7500, 6000]
})
water_data['Demand Per Capita'] = water_data.apply(water_demand, axis=1)
print(water_data)Get Manoj Singh’s stories in your inbox
Join Medium for free to get updates from this writer.
Example 2: Identifying Risk Levels for Reservoirs
def reservoir_risk(row):
if row['Storage (m3)'] < 50000:
return 'High Risk'
elif row['Storage (m3)'] < 100000:
return 'Medium Risk'
else:
return 'Low Risk'
reservoir_data = pd.DataFrame({
'Reservoir': ['X', 'Y', 'Z'],
'Storage (m3)': [45000, 80000, 120000]
})
reservoir_data['Risk Level'] = reservoir_data.apply(reservoir_risk, axis=1)
print(reservoir_data)4. Lambda Functions in Python
Lambda functions are anonymous, one-line functions defined using the lambda keyword. They’re often used for short, simple operations without the need to explicitly define a function using def.
Syntax:
lambda arguments: expressionApplications in Water Resources
Example 1: Quick Unit Conversion
rainfall_inches = [2.0, 1.8, 3.2, 0.5]
rainfall_mm = list(map(lambda x: x * 25.4, rainfall_inches))
print(rainfall_mm) # Output: [50.8, 45.72, 81.28, 12.7]Example 2: Filtering Data
flow_data = [850, 1200, 950, 1100]
flood_warnings = list(filter(lambda x: x > 1000, flow_data))
print(flood_warnings) # Output: [1200, 1100]Example 3: Sorting a DataFrame by Calculated Values
import pandas as pd
data = pd.DataFrame({
'Region': ['A', 'B', 'C'],
'Flow (m3/s)': [120, 95, 150],
'Area (km2)': [50, 60, 40]
})
data['Flow Density'] = data.apply(lambda row: row['Flow (m3/s)'] / row['Area (km2)'], axis=1)
data_sorted = data.sort_values(by='Flow Density', ascending=False)
print(data_sorted)Example 4: Generating Labels for Plotting
labels = list(map(lambda x: f'Region {x}', ['A', 'B', 'C']))
print(labels) # Output: ['Region A', 'Region B', 'Region C']Example 5: Combining Columns for Analysis
data['Summary'] = data.apply(lambda row: f"{row['Region']}: {row['Flow Density']:.2f} m3/s/km2", axis=1)
print(data['Summary'])5. JavaScript .map() in Google Earth Engine (GEE)
Google Earth Engine (GEE) leverages JavaScript’s .map() function to perform operations over collections, such as image collections or feature collections.
Syntax:
collection.map(function)Applications in Water Resources
Example 1: Calculating NDWI for Image Collection
var calculateNDWI = function(image) {
return image.addBands(
image.normalizedDifference(['B3', 'B8']).rename('NDWI')
);
};
var sentinelImages = ee.ImageCollection('COPERNICUS/S2')
.filterBounds(region)
.map(calculateNDWI);Example 2: Filtering Flooded Areas
var classifyFlood = function(feature) {
var flooded = feature.get('water_level') > 1;
return feature.set({'Flooded': flooded});
};
var waterData = ee.FeatureCollection('users/water_data')
.map(classifyFlood);Conclusion
By understanding the versatility of .map(), .apply(), map(), and lambda functions across Python, Pandas, and JavaScript, you can unlock powerful data transformation and analysis capabilities. These functions are particularly valuable in water resources engineering, where handling large datasets efficiently is key. Whether you're converting units, classifying risks, sorting data, or processing satellite imagery, these tools will streamline your workflow and enhance your analytical capabilities.
