Tricks to Import Huge Amount of Data Into Odoo in Seconds

Name: Tricks to Import Huge Amount of Data Into Odoo in Seconds
Rating: 4.3 (49 reviews)

Regularly import million of external records FAST!

Created byAkhmad Daniel Sembiring

Last updated 6/2023

English

What you'll learn

Techniques to import external data into Odoo fast
Using XMLRPC Standard
Using Direct SQL
Using Stored Procedure PL/PGSQL
Automating the Import Process
Filtering Record to Import

Course content

8 sections • 44 lectures • 3h 27m total length

Introduction0:43
Download Resources0:03
Background3:15
Explore practical methods to import large data sets into Odoo, from staging databases and CSP files to automated add-ons and engine-based integrations for faster performance.
Case Study2:38
Restore Sample Database3:38
Datasource Analysis2:02
Analyze the datasource structure of employees, salaries, and departments to map one-to-many and one-to-one relations, including a junction table for employee and department links before importing large datasets into Odoo.
Datasource Mapping2:10
Initial Specification0:50

Direct SQL Explanation1:52
Install MySQL Connector for Python1:26
Create Addon2:15
Create Wizard Model3:45
Create a wizard model in Odoo to import a large number of employee records quickly, detailing setup of the Python module, action method, and user interface flow.
Create Wizard View9:52
Test and Install Addon1:44
Test and install the addon in Odoo by configuring the module grid, verifying no errors, and iterating on the employee configuration and related menus as you define the method.
The Import Logic6:32
Establish MySQL Connection3:29
Establish a database connection by importing the skill connector package, defining host, user, password, and database constants, and using the connection handler to connect to the employees database.
Pull Department Data1:48
Push Department to Odoo4:29
Pull Employee Data5:05
Push Employee to Odoo13:57
Method 2 Result2:27

Store Procedure Explanation5:09
Modify Create Department Method9:07
Import Department Stored Function15:08
Import Department Stored Function Part 28:25
Modify Create Employee Method4:36
Create Employee Stored Function14:38
Test the Create Employee Stored Function1:29
Integrate, Upgrade Addon and Run5:37
Method 3 Test Result1:38
Showcases method 3 test result by importing 300,000 employee records into odoo in about one minute, highlighting a significant speed improvement and timing checks.

Requirements

Odoo Addon Fundamental
Odoo XMLRPC Fundamental
Python Language
SQL Fundamental

Description

Do you have Odoo implementation cases to regularly import a huge amount of data from external data source like MS SQL, Oracle, MySQL, etc? I meant, very huge, like million of invoices or journal entries record per day ?

Using the common standard way, the Odoo's XMLRPC and ORM, this process can take hours, if not timeout, while we are taking a few cups of coffee as Odoo suggests :)

This course explains in detail about techniques to improve the data import performance, starting from data source query, processing in Python, and processing in Odoo Database (PostgreSQL) to achieve a drastic performance increase, that was previously taking more than 10 hours to only seconds! Yes, seconds.. how can it be ? Is that possible ?

YES

And we have implemented this technique in many implementation cases where Odoo's default mechanism is no more capable to handle that million of records per day!

Enjoy !

Who this course is for:

Odoo Developer
Odoo Consultant
Odoo Implementor
Odoo Official Partner

Tricks to Import Huge Amount of Data Into Odoo in Seconds

What you'll learn

Explore related topics

Course content

Introduction8 lectures • 15min

Method 1: PHP Middleman7 lectures • 32min

Method 2: Direct SQL13 lectures • 59min

Method 3: Stored Procedure9 lectures • 1hr 6min

Filtering Record2 lectures • 14min

Update or Create Record2 lectures • 8min

Automating the Process2 lectures • 10min

Conclusion1 lecture • 4min

Requirements

Description

Who this course is for: