Let's Build Startup for Remote Drone Operations, 4G & Video

Outline the big picture and environment setup for remote drone operations, linking a local server, Raspberry Pi with Python, and JavaScript frontend to an Ubuntu 16.04 simulator.

Install our software-in-the-loop simulator on Ubuntu 22 with VMware Workstation Player 17, replacing VirtualBox; clone the repository, install prerequisites, build Arducopter, and run the Python simulator.

Set up a Raspberry Pi with a camera on a wired network. Use a simulator for development, then mount it on a frame and connect to telemetry, drone, and cloud.

Set up a java project in Visual Studio Code, install JDK 11 and Java 8, and configure a Gradle-based springboard project with WebSocket support.

Configure java logging with logback for our application, using asynchronous console and file appenders, reading environment variables from yaml, and setting per-package log levels.

Create the initial Biton Python app in VS Code, set up logging, initialize a git repository with a readme, and prepare a Raspberry Pi deployment with a log file.

Configure your Python application to run as a Linux service on a Raspberry Pi by creating a dynamic systemd service file, handling root paths and logging to console and files.

Learn to set up a Python configuration reader to manage drone, cloud, and video settings, parse and validate config files, log runs, and handle errors on Raspberry Pi deployments.

Create a video streaming workflow that reads camera frames, encodes them as JPEG buffers, and sends each frame as a UDP message with drone IDs to a Java job application.

Commit changes to git after coding, stage files, and write meaningful messages to track edits and line changes; review commit history to monitor progress and support team collaboration.

Develop a video page that receives back-end variables (host, drone ID, video endpoint) and activates a WebSocket video stream, updating the image element with live feed.

Implement a JavaScript video stream client using a WebSocket to receive base64 JPEG frames from the backend, update the image source per frame, and register the active drone ID.

Add a /v/{droneId} video endpoint to the base controller, populate the model with IP, drone id, and video path, and return the video template to serve the feed.

Configure a Spring-based configurator to read the YAML settings for drone video endpoints and UDP payload structure, injecting them for runtime access on the video page.

Configure the video websocket endpoint in the backend by implementing websocket configuration and a handler, register it with the websocket registry, and manage drone sessions with a video stream manager.

Develop the video stream manager to facilitate drone-to-page streaming, mapping each drone to multiple WebSocket sessions via a thread-safe hash set for concurrent access.

Implement a Java video stream manager that opens a UDP data socket, uses a single-thread executor, and dispatches incoming data to WebSocket sessions, with Spring integration.

Implement the core video stream manager: read UDP packets, extract drone ID and JPEG data, broadcast frames to all registered WebSocket sessions, and maintain resilience with try-catch and safe shutdown.

Build and run a remote drone video streaming app using a Raspberry Pi, WebSocket sessions, and a video stream manager, while discussing tests, public methods, and interfaces for prototyping.

Finalize the index page of the single-page app with a world map powered by Google Maps, a drone list, and video sections driven by dynamic JavaScript UI components.

Configure the index page backend endpoint by binding YAML values to model attributes via a reader. Expose the default drone speed and altitude to the front end for JavaScript.

Learn vanilla JavaScript for this project without Angular or React, using a single library called January for selectors to implement dynamic behavior and reusable UI components.

Initialize a JavaScript app for remote drone operations by setting up drone data, a world map, and markers for mission points, then enable keyboard controls and periodic system data updates.

Implement the update system data method to fetch latest drone data via a GET request, log the response, and center the map on the selected drone's coordinates.

Define the drone info data transfer object to formalize back-end to front-end communication, detailing an immutable object with latitude, longitude, speed, altitude, battery, and state, using Lombok to generate boilerplate.

Learn how to load drones data via an asynchronous ajax call, update drone positions on a map, manage online status, and render dynamic drone user interface.

Develop a JavaScript frontend drone abstraction that manages a drone object, video streaming, WebSocket control, and real-time position on a Google Maps world map.

Create a Google Maps marker to capture drone location on click, updating latitude and longitude, and tie the marker to the selected drawing with a point id and info window.

Learn how to initialize drone controls, generate ui buttons, and map keyboard events to drone commands. The lecture explains sending codes to the backend and ultimately to the Raspberry Pi.

Explore rendering the map point data component, configuring marker polling data, and updating or removing drone point data via a form-driven info window with height, speed, and action settings.

Develop and render a dynamic drone UI by implementing a UI components method that builds control panels, toggles between map and free views, and starts video feeds via WebSocket.

Explore how to implement backend endpoints to update system data and drone status, migrate inline styles to a separate css file, and use mock data to visualize front-end integration.

Build a mock json endpoint that returns drone status and location data for two drones, using a spring controller and json conversion to send json to the front end.

Implement and test start mission and command endpoints, sending drone id and JSON points, log received data, and validate front-end and back-end contracts for reliable data exchange.

Add a live video feed to the UI front page by streaming from the Raspberry Pi via WebSocket, updating the image source with JPEG data for the selected drone.

Implement the backend rest controller for remote drone operations, wiring endpoints to a control manager that handles system info, mission start, and command delivery with serialized data points.

Implement a control manager that opens a server socket, accepts Raspberry Pi connections, and registers drone handlers in a map by drone ID, enabling TCP/IP data exchange and lifecycle management.

Explore java dronehandler initialization, wiring a control manager and drone socket with blocking queues for non-blocking commands and a producer-consumer buffer to manage network data exchange.

Explore the drone handler receiver and sender threads, where a runnable polls network data into a drone info object, tracks the last update time, and writes commands from a queue.

Implement drone handler behaviors for sending mission data and commands, reading the latest drone status, and buffering messages through a network pipeline using the data mopper.

Learn protobuf basics and generate Java and Python libraries from a proto file to encode drone telemetry into bytes over tcp/ip 4g, using command data, drone data, and mission data.

Explore how a data mapper converts domain objects to protobuf buffers and back, producing compact byte streams with command codes, payloads, mission data, and data points.

Implement a simple Java network protocol by encoding message size in a four-byte header, sending a combined header and body, and reading data byte-by-byte to ensure complete messages.

Run and test a Java backend that connects phone data to a Raspberry Pi, verifies builds, runs locally, streams video, and executes drone mission data via a simulator.

Develop a Python app on a Raspberry Pi that connects to a flight controller via a modeling protocol, runs as Linux service, and streams video for simulator or real hardware.

Bootstraps a python drone app by loading configuration, initializing logging, and managing a watchdog-driven connection loop for a simulator or real drone, with socket communication and video streaming.

Explore how to implement a drone connection watchdog in Python by extending a thread, overriding its run method, and continuously monitoring internet status with reconnection logic.

Describe how a Python data receiver thread extends a threat object to continuously read network messages, deserialize protobuf commands, and dispatch drone actions via a cross-language command pipeline.

Explore how a drone object abstracts simulator or real hardware, exposing macro level control and serializing telemetry via dronekit and proto data for Movielink and Mudlick interfaces.

Map domain objects to Protobuf data, serialize to bytes, and prepend a four-byte length header to form and transmit network messages, then decode received data back to domain objects.

Introduce a Python drone control panel object that maps command codes to actions, with separate engine and servo threads on Raspberry Pi for precision control and mission activation.

Implement an engine thread that maintains constant speed and direction from a control panel, issues periodic 1.5-second updates, supports immediate changes, emergency kill, and rotation commands.

Develop a Raspberry Pi camera servo controller using wiringpi with a dedicated thread that outputs PWM to sweep a 180-degree angle (60-220) safely.

Learn the complete multithreaded drone control flow, from configuration and drone object creation to connection watchdog, data receiver commands, telemetry streaming, and resilient Raspberry Pi video processing.

Demonstrates a full distributed drone application with two Raspberry Pi drones, simulators on virtual machines, deploying a Java backend, launching missions, streaming video, and handling reconnection and return-to-launch.

Embark on a maiden flight for territory exploration, take off amid wind gusts, and have fun while securing initial black boxes and all project files for remote drone operations.