FarmSentinel

FarmSentinel
Your 24/7 Farm Monitor Against Crop-Damaging Tiny Pests
  
  
Project Brief
  
An automated monitoring solution based on Microsoft's FarmVibes project series. Using computer vision, we built an cost-effective hardware-software system.
Sponsored By
My Role
  
Team Lead
UI/UX Designer
Front & Back-end Developer
Project Type
  
To B Team Project
Sponsored by
Microsoft FarmVibes
Time Frame
  
March - June 2024
Tools Used
  
VScode, React Framework, Flask Server, Figma, Adobe Photoshop, Protopie
Problem Statement
  
  
Pea Weevils, a tiny pest measuring 0.3–0.5 cm, devastate pea crops, slashing their value by over 70% each spring and summer.

For Andrew Nelson, who runs Washington’s largest pea farm, the battle is relentless. The weevils’ rapid reproduction forces his team to patrol fields daily during spring & summer, draining time, labor, and resources. The need for an automated monitoring solution has never been more urgent...
  
  

Pain point 1

  
Pest management is time-consuming and labor consuming.
  
  

Pain point 2

  
Limited labor availability and high labor costs, especially for consistent pest monitoring.
  
  

Pain point 3

  
Field sensors are expensive to buy and maintain, especially for a 7500-acre farm.
Impact

83%

Time on tiny pest saved for farmers like Andrew

10k+

Labor cost expexted to be saved yearly with extra surplus from crop

1

Paper published at the IEEE IoTaIS 2024.
Quick View of Deliverables 🌟
  
  
  
  
How Does The System Work?
  
  
  
  
Feature 1 - Unified Dashboard
  
This is the primary interface users engage with most frequently. Given that a significant portion of our users are over 35, including a notable number of seniors, we designed the Dashboard (set as the Homepage) to visually and concisely display all key data collected from the field in real time.

It provides a clear, at-a-glance view of essential field data and pest status. Call-to-actions (CTA), such as viewing device details and accessing the pest Warning List, are prominently displayed to minimize cognitive load and ensure ease of use.
  
  
  
  
Feature 2 - Data Analysis
  
  
  
  
Feature 3 - Device Management
  
Rather than driving all day between fields, users can simply click on devices with pest detections from a list or map to view the latest photos and pest counts. They can also access all related photos and data, adjust detection intervals, and remotely control the device to capture real-time images.
Design System
  
  
  
  
  

Challenges

  
  
  
  
We constantly face diverse challenges, such as:
1. Selecting and testing components, managing timelines and budgets, and optimizing cost-performance.
2. Deciding on approaches like computer vision vs. machine learning, pest elimination capabilities, and data transmission via LoRa or cellular.
3. Learning new skills, such as Flask server development and database management.
4. Balancing usability and aesthetics to ensure intuitive design, even for older users.
...

We kept exploring, testing and pivioting until we found the best solution with the resources we had.
  
  
  
  
  
  

A/B Testings & Decisions

  
  
  
  
We conducted usability testing for all features, both software and hardware. We began with simulations and then field test.

Starting with a Figma low-fidelity prototype, we progressed to mid-fidelity and finally high-fidelity designs.

To optimize time, back-end development began during the low-fi phase, and front-end module development started during the hi-fi phase. After completing the hi-fi design, we refined the front-end with CSS styling while retaining the core module structure.
  
  
  
  
  
  
  
  
Reflection
  
  
  
- Challenge Navigating Negotiations and Securing the Project
  
This sponsored course project taught me invaluable lessons, particularly in communication and negotiation.
At the outset of the semester, our team, composed solely of three students with design backgrounds, faced stiff competition from groups with stronger CS expertise, all vying for the Microsoft FarmVibes collaboration. Given the project’s heavy emphasis on IoT devices, we were at a clear disadvantage. However, determined to seize this learning opportunity, I calmly assessed the situation and quickly mapped out the project's framework and technical roadmap. Leveraging my communication skills, I initiated negotiations with the project supervisor, articulating our clear vision and technical roadmap. Through strategic discussions, I turned our perceived weaknesses into strengths, and despite the odds, we emerged victorious, securing the project. This experience was a significant test of my abilities, as I had less than 30 minutes to strategize and execute the negotiation plan, ultimately leading to our success.
  
  
  
- Overcoming Challenges and Leading the Team
  
The project’s 10-week timeframe was extremely tight, and we were tasked with solving complex problems. We had to start from scratch, learning everything from Raspberry Pi programming to database setup, front-end development, and data transmission. I invested considerable time in developing a streamlined, AI-assisted learning & working flow that allowed for rapid testing and iteration of our hardware. As the team leader, I also organized regular work sessions to address challenges collectively, fostering a supportive environment that kept the team motivated. Despite the daily hurdles, including persistent software bugs, no one gave up. Time management was another major challenge, but we managed it well. Our sponsor was highly impressed with our dedication and outcomes, and we were the only team to have our paper accepted for the GHTC conference. Our project exceeded expectations in both scope and completeness, earning us second place in the final evaluation among all sponsored projects in our department. Reflecting on this journey, I am proud of my relentless learning ability and my "Make it happen" mindset, which drove me to seek external help and expand my knowledge, successfully navigating the entire product development lifecycle. This experience also deepened my appreciation for how AI can significantly empower designers and engineers.
  
  
  
  
- Project Limitations and Future Directions
  
Honestly, when we tested the third-generation prototype in the field, it didn’t perform as well as we had anticipated. Several factors contributed to this outcome. First, due to budget constraints, the IR sensors we used lacked precision. Reducing the monitoring frequency to meet energy consumption requirements further decreased sensor sensitivity, leading to some inaccuracies (as detailed in Experiment 3 of our paper). Second, material limitations meant that our prototype did not adequately address waterproofing, wind resistance, and high-temperature endurance. Third, the cost of the third-generation prototype was around $200 per unit, which needs to be reduced. Fortunately, by switching to a cheaper motorized Pi camera and a lighter microprocessor, we could potentially lower costs by nearly 50%. Despite these challenges, our project explored innovative solutions that were highly original. In an era dominated by Machine Learning models and LLMs, our decision to use a combination of DSP and computer vision, which are more cost-effective and user-friendly, was validated by the GHTC committee’s recognition. This project has laid a solid foundation for future iterations and improvements.
  
Future Plan
  
Enhancing Device Structure
Improve the physical design of the devices to ensure they are more robust and better suited for the challenging environments of large-scale farming.
  
Optimizing Front-End and Back-End Code
Refine the software architecture, enhancing both performance and cybersecurity measures to protect sensitive agricultural data.
  
Upgrading Material Quality
Prototype with more durable materials that can withstand the varied and often harsh conditions found in agricultural settings, ensuring consistent and reliable operation.