Get started
Work

Lightweight, Pre-trained Transformers for Remote Sensing Timeseries

Year: 2023
Type: preprint
Abstract: Machine learning methods for satellite data have a range of societally relevant applications, but labels used to train models can be difficult or impossible to acquire. Self-supervision is a natural solution in settings with limited labeled data, but current self-supervised models for satellite data fail to take advantage of the characteristics of that data, including the temporal dimension (which is critical for many applications, such as monitoring crop growth) and availability of data from many complementary sensors (which can significantly improve a model's predictive performance). We present Presto (the Pretrained Remote Sensing Transformer), a model pre-trained on remote sensing pixel-timeseries data. By designing Presto specifically for remote sensing data, we can create a significantly smaller but performant model. Presto excels at a wide variety of globally distributed remote sensing tasks and performs competitively with much larger models while requiring far less compute. Presto can be used for transfer learning or as a feature extractor for simple models, enabling efficient deployment at scale.
DOI: 10.48550/arxiv.2304.14065
Authors: Tseng, Gabriel, Cartuyvels, Ruben, Zvonkov, Ivan, Purohit, Mirali, Rolnick, David, Kerner, Hannah
Language: English
Open Access status: green
Award: THIS PROPOSED FOOD SECURITY AND AGRICULTURE CONSORTIUM (FSAC) WILL ADDRESS VARIOUS ASPECTS OF THE APPLICATION OF EARTH OBSERVATIONS (EO) TO DOMESTIC AND GLOBAL AGRICULTURE AND FOOD SECURITY THROUGH AN UNPRECEDENTED GROUP OF RESEARCH AND OPERATIONAL PARTNERS. WE HAVE ASSEMBLED A CONSORTIUM OF PRINCIPALS DRAWN FROM LEADING ORGANIZATIONS IN THE U.S. AND AROUND THE WORLD THAT USE EO AND DERIVED INFORMATION TO ADVANCE AGRICULTURE AND FOOD SECURITY. THE PROPOSED FSAC ANCHORS A BROAD NETWORK OF PARTNERS DRAWN FROM UNIVERSITIES GOVERNMENT AGENCIES AND RESEARCH ORGANIZATIONS NGOS INTERNATIONAL ORGANIZATIONS AND THE PRIVATE SECTOR REPRESENTING END USERS INFORMATION AND SERVICE PROVIDERS AND APPLICATIONS RESEARCHERS. FSAC PARTNERS LEAD THE WORLD IN ENGAGING END USERS IN USING EO-DERIVED INFORMATION FOR AGRICULTURE. THE FSAC EXPANDS ON EXISTING AND PAST NASA INVESTMENTS AND IS SPECIFICALLY DESIGNED TO AMPLIFY THE BENEFITS BROADLY THROUGH PUBLIC AND PRIVATE SECTORS IN THE US AND INTERNATIONALLY. LEVERAGING OUR CURRENT AND PAST WORK TO BUILD THE GEO G20 GLOBAL AGRICULTURAL MONITORING (GEOGLAM) INITIATIVE THE CURRENT PROPOSAL IS ESSENTIAL TO THE CONTINUED IMPLEMENTATION AND DEVELOPMENT OF GEOGLAM WHICH HAS ACHIEVED UNPRECEDENTED SUCCESS IN ENGAGING END-USERS INVOLVED WITH CROP PRODUCTION MONITORING AND FORECASTING. THROUGH THIS PROPOSAL THE CROP MONITORS LED BY UMD WILL CONTINUE TO DELIVER EO-DERIVED INFORMATION TO THE AGRICULTURAL MARKET INFORMATION SYSTEM (AMIS) AN INTERNATIONAL BODY OF CHIEF ECONOMISTS FROM MAJOR CROP EXPORT COUNTRIES AND PROVIDE A MONTHLY BULLETIN OF CROP CONDITIONS FOR FOOD INSECURE COUNTRIES WITH AN EXPANDING GROUP OF INTERNATIONAL AND NATIONAL FOOD SECURITY ORGANIZATIONS. THE FSAC END USER ENGAGEMENT WILL BE BROAD - ADDRESSING NOT ONLY INTERNATIONAL CROP PRODUCTION FORECASTS MARKETS AND TRADE ALONGSIDE REGIONAL/NATIONAL FOOD SECURITY EARLY WARNING DISASTER RESPONSE AND POLICY - BUT ALSO DOMESTIC COMMERCIAL AGRICULTURE AND FARM MANAGEMENT AND SMALLHOLDER FARM MANAGEMENT RESILIENCE AND MICRO-INSURANCE. IT WILL INCLUDE STRATEGIC SYSTEMATIZED ENGAGEMENT OF NON-TRADITIONAL USERS OF NASA DATA (INSURANCE INDUSTRY HUMANITARIAN ORGANIZATIONS AND INTELLIGENCE). THE FSAC MEMBERS WERE SELECTED STRATEGICALLY FOR THEIR EXPERTISE AND RELATIONSHIPS WITH A BROAD RANGE OF USER-GROUPS. OUR APPLICATIONS R&D WILL FOCUS ON CROP CONDITION MONITORING AND RISK ASSESSMENT CROPLAND CROP TYPE AND AREA MAPPING CROP YIELD AND PRODUCTION MODELING AND FORECASTING AND LINKING EO PRODUCTS TO SOCIO-ECONOMIC DATA TO ADDRESS FOOD SECURITY QUESTIONS. THE RESULTING SUITE OF INNOVATIVE PRODUCTS AND SERVICES WILL COMBINE EO PRODUCTS FROM DIFFERENT SENSORS WITH METEOROLOGICAL GROUND AND SURVEY DATA. OTHER MAJOR WORK ELEMENTS WILL INCLUDE: THE ECONOMIC AND SOCIAL VALUATION OF THE BENEFITS OF EO THROUGH ECONOMIC ANALYSIS OF EO FOR PRODUCTION FORECASTING AND REPRESENTATIVE END USER CASE STUDIES; LINKAGE TO NEW NASA SENSORS ENHANCING THE EARLY ADOPTERS PROGRAM; DEVELOPMENT OF AN FSAC PORTAL FOR INFORMATION COLLECTION DISSEMINATION AND INNOVATIVE COMBINATION; AND A PROFESSIONAL COMMUNICATIONS INITIATIVE GIVING BROAD OUTREACH FOR THE PROGRAM. THE DOMESTIC AGRICULTURE COMPONENT OF THE PROPOSAL WILL TEST NEW EO-BASED APPLICATIONS TO IMPROVE FARM MANAGEMENT AND HELP USDA NASS IMPROVE THE RELIABILITY OF ITS US STATISTICS (AREA/YIELD). FSAC WILL HOLD A SERIES OF THEMATIC COMMUNITY WORKSHOPS IN THE US ESTABLISHING A US GEOGLAM COMMUNITY OF PRACTICE (COP) TO COMPLEMENT THE INTERNATIONAL COP. THESE ACTIVITIES TOGETHER WILL DELIVER NASA AN OUTSTANDING COORDINATED AND WELL-BALANCED PROGRAM APPLYING EO TO FOOD SECURITY AND AGRICULTURE WITH STRONG CONNECTIONS TO RELEVANT NASA PROGRAMS SCIENCE TEAMS AND NEW MISSIONS. A GOAL OF THE FSAC IN ADDITION TO RESPONDING TO THE RFP IS TO EXPAND ITS ACTIVITIES PARTNERSHIPS AND COLLABORATIONS WITH NON-NASA FUNDING SUSTAINING THE CONSORTIUM AND ITS FUNCTIONS BEYOND THE 5-YEAR FUNDING PERIOD.
OpenAlex ID: W4367365754
Cites:
Cited by: 37
Related to: 10