robot/esp32-opencv
1
0
Fork 0
mirror of https://gitcode.com/gh_mirrors/es/esp32-opencv.git synced 2025-08-14 01:57:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

Big refactor of the doc

Browse Source
This commit is contained in:
Joachim
2020-05-07 13:18:35 +02:00
parent 403ffc7703
commit 285ba56370
3 changed files with 177 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files

187
README.rst
View File

@ -138,18 +138,75 @@ The first way is to simply get the pre-built OpenCV library in `esp32/lib/`_ fol
Fast way:
---------
The second way is by using the script in build_opencv_for_esp32.sh_. This script automatically compiles OpenCV from this repository sources, and install the needed files into the desired project. It can tweaked as needed to add and remove some parts.
The script has 2 arguments. The first is the path to the ``toolchain-esp32.cmake`` (default is ``$HOME/esp/esp-idf/tools/cmake/toolchain-esp32.cmake``), and the second is the path where the OpenCV library is installed (default is in esp32/lib).
The second way is by using the script in build_opencv_for_esp32.sh_. This script automatically compiles OpenCV from this repository sources, and install the needed files into the desired project. It can be tweaked as needed to add and remove some parts (see `esp32/doc/build_configurations.md`_).
.. _build_opencv_for_esp32.sh: esp32/scripts/build_opencv_for_esp32.sh
.. _`esp32/doc/build_configurations.md`: esp32/doc/build_configurations.md
The script has 2 arguments. The first is the path to the ``toolchain-esp32.cmake`` (default is ``$HOME/esp/esp-idf/tools/cmake/toolchain-esp32.cmake``), and the second is the path where the OpenCV library is installed (default is in ``./esp32/lib``).
Detailed way:
-------------
The last way explains all the commands and modifications done to be able to compile and run OpenCV on the ESP32. The detailed procedure is in detailed_build_procedure.md_.
The last way explains all the commands and modifications done to be able to compile and run OpenCV on the ESP32. The detailed procedure is in `esp32/doc/detailed_build_procedure.md`_.
.. _`esp32/doc/detailed_build_procedure.md`: esp32/doc/detailed_build_procedure.md
Compiling esp-idf project using OpenCV:
=======================================
When the OpenCV library is cross-compiled, we have in result ``*.a`` files located in ``build/lib`` folder. We now want to try to compile an example project using OpenCV on the esp32. A basic example of esp-idf project can be found in `esp32/examples/esp_opencv_basic/`_. This project simply creates an OpenCV matrix, fill it with values and prints it on the console. It's only purpose is to test the installation.
.. _`esp32/examples/esp_opencv_basic/`: esp32/examples/esp_opencv_basic/
Esp-idf environment uses cmake and is separated in components. Because OpenCV libs were compiled outside this example project, we use the pre-built library functionality of esp-idf (https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/build-system.html#using-prebuilt-libraries-with-components).
Here are the things done to add the OpenCV library to the project:
- Copy the ``opencv/`` folder (from `esp32/lib/`_) into your project's ``main/`` component's folder. It contains the generated libraries (``libade.a``, ``libopencv_core.a``, ``libopencv_imgproc.a`` and ``libopencv_imgcodecs.a``) and some needed header files (e.g. ``cvconfig.h``, ``opencv_modules.hpp``, ``core.hpp``, ``core/`` folder, etc..).
.. _`esp32/lib/`: esp32/lib/
- Link the libraries to the project by modifying the ``CMakeList.txt`` of the ``main`` project's component as below :
.. code:: cmake
idf_component_register(
SRC main.cpp
INCLUDE_DIRS ./opencv
)
add_prebuilt_library(opencv_imgcodecs "opencv/libopencv_imgcodecs.a")
add_prebuilt_library(libpng "opencv/3rdparty/liblibpng.a")
add_prebuilt_library(libzlib "opencv/3rdparty/libzlib.a")
add_prebuilt_library(opencv_imgproc "opencv/libopencv_imgproc.a")
add_prebuilt_library(opencv_core "opencv/libopencv_core.a")
add_prebuilt_library(ade "opencv/libade.a")
set(IMGCODEC_DEP libpng libzlib)
target_link_libraries(opencv_imgcodecs INTERFACE ${IMGCODEC_DEP})
set(CORE_DEP libzlib)
target_link_libraries(opencv_core INTERFACE ${CORE_DEP})
set(OPENCV_DEP opencv_imgcodecs opencv_imgproc opencv_core)
target_link_libraries(${COMPONENT_LIB} ${OPENCV_DEP})
- Finally, include the OpenCV headers needed into your source files:
.. code:: c++
#include "opencv2/core.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
.. _detailed_build_procedure.md: esp32/doc/detailed_build_procedure.md
Get project RAM and Flash usages
@ -167,6 +224,35 @@ At compilation time:
- The file ``build/<project-name>.map`` is also very useful. It indicates the memory mapping of the variables and can be used to find big variables in the application.
- The commands ``idf.py size``, ``idf.py size-files`` and ``idf.py size-components`` are very useful to see the memory segments usage. They show more precise information, and also per file usage. For instance with the `esp32/examples/esp_opencv_basic/`_ project, the size used is :
.. _`esp32/examples/esp_opencv_basic/`: esp32/examples/esp_opencv_basic/
.. code:: shell
Total sizes:
DRAM .data size: 21168 bytes
DRAM .bss size: 13280 bytes
Used static DRAM: 34448 bytes ( 146288 available, 19.1% used)
Used static IRAM: 61849 bytes ( 69223 available, 47.2% used)
Flash code: 843403 bytes
Flash rodata: 246200 bytes
Total image size:~1185900 bytes (.bin may be padded larger)
And for the `esp32/examples/esp_opencv_tests/`_ project, the size used is:
.. _`esp32/examples/esp_opencv_tests/`: esp32/examples/esp_opencv_tests/
.. code:: shell
Total sizes:
DRAM .data size: 31812 bytes
DRAM .bss size: 14096 bytes
Used static DRAM: 45908 bytes ( 134828 available, 25.4% used)
Used static IRAM: 63741 bytes ( 67331 available, 48.6% used)
Flash code: 1373491 bytes
Flash rodata: 347440 bytes
Total image size:~1830580 bytes (.bin may be padded larger)
At run time:
------------
@ -179,76 +265,35 @@ At run time:
ESP_LOGI(TAG, "heap left: %d Bytes", esp_get_free_heap_size());
Adding images codecs support
============================
DRAM region overflow
====================
Things done to read/writes images in JPEG, PNG, etc..
Depending on which part of the OpenCV library is used, some big static variables can be present and the static DRAM can be overflowed. The following errors can appear:
PNG
---
- dram overflow
.. code:: shell
- Remove ``-DWITH_PNG=OFF`` and add ``-DBUILD_PNG=ON`` and ``-DBUILD_ZLIB=ON`` of the cmake command
.dram0.bss will not fit in region dram0_0_seg ; region 'dram0_0_seg' overflowed by N bytes
- The lib ``opencv_imgcodecs.a`` build pass
The library is compiled in the ``3rdparty/`` folder. Copy this folder into the esp32 example project folder.
JPEG
----
- Remove ``-DWITH_JPEG=OFF`` and add ``-DBUILD_JPEG=ON`` of the cmake command
- Problem at compilation time. TODO
Adding parallel support
=======================
TODO
Removing OpenCV unnecessary parts
=================================
Opencv is quite big, even when compiling only the core, imgproc and imgcodec modules. Because the ESP32 has limited resources, it is a good idea to remove some parts of opencv that are in most cases not used.
TODO: List the modules functionalities and what is kept or not
Core module:
------------
Imgproc module:
---------------
Colorspaces
^^^^^^^^^^^
Opencv supports multiple colorspaces (RGB, BGR, RGB565, RGBA, CIELAB, CIEXYZ, Luv, YUV, HSV, HLS, YCrCb, Bayer, Gray). All these colorspaces are not mandatory for an embedded system, so some are removed.
- ``color_lab.cpp``: This file contains conversion for CIELAB and CIEXYZ (https://en.wikipedia.org/wiki/CIELAB_color_space). The conversion tables takes a lot of space in the .bss segment (~88kB) , which is already overflowing. Here are the steps done to disable this code:
- Move ``color_lab.cpp`` to ``color_lab.cpp.bak``
- In ``color.hpp`` disable :
.. code:: c++
// todo
- SHA256 digest overwrite
.. code::shell
A fatal error occurred: Contents of segment at SHA256 digest offset 0xb0 are not all zero. Refusing to overwrite.
The DRAM is the internal RAM section containing data. From the linker script ``esp-idf/components/esp32/ld/esp32.ld``, the ``dram_0_0_seg`` region has a size of ``0x2c200``, which corresponds to around ``180kB``. Due to some fixed RAM addresses used by the ESP32 ROM, there is a limit on the amount which can be statically allocated at compile time (see https://esp32.com/viewtopic.php?t=6699). To prevent this, there are some solutions:
- If not used, disable Bluetooth and Trace Memory features from the menuconfig. Bluetooth stack uses 64kB and Trace Memory 16kB or 32kB (see https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/general-notes.html#dram-data-ram)
- In the menuconfig, the following options can also reduce internal DRAM usage:
- In Component Config -> ESP32-specific -> Support for external, SPI-connected RAM -> SPI RAM config, enable :
- "Try to allocate memories of WiFi and LWIP in SPIRAM firstly. If failed, allocate internal memory"
- "Allow .bss segment placed in external memory"
- Search for big static array that could be stored in external RAM
- In ``build/<project_name.map`` file of your project, look under the section ``.dram0.bss`` for big arrays
- ``idf.py size-files`` command is also useful
When big arrays are found, either apply the macro ``EXT_RAM_ATTR`` on them (only with option .bss segment placed in external memory enabled), or initialize them on the heap at runtime.
- In ``color.cpp`` disable:
.. code:: c++
// todo
- todo

53
esp32/doc/build_configurations.md Normal file
View File

@ -0,0 +1,53 @@
# Build configurations
This doc details some build configurations possible.
Images codecs support
----------------------------
To add/remove image codecs support, the following modifications are done.
### PNG
- Remove ``-DWITH_PNG=OFF`` and add ``-DBUILD_PNG=ON`` and ``-DBUILD_ZLIB=ON`` from the cmake command
- The lib ``opencv_imgcodecs.a`` build pass
The library is compiled in the ``3rdparty/`` folder. Copy this folder into the esp32 example project folder.
### JPEG
- Remove ``-DWITH_JPEG=OFF`` and add ``-DBUILD_JPEG=ON`` of the cmake command
- Problem at compilation time. Doesn't support JPEG for now (TODO).
## Additional OpenCV modules
In addition to the `core`, `imgproc` and `imgcodecs` modules, the following were also tested:
* `features2d`: The cross-compilation didn't cause issues. The following functionalities were used:
* AKAZE features detector/descriptor
* Matching method
* `calib3d`: The cross-compilation didn't cause issues. The following functionalities were used:
* undistortPoints method
* findHomography method
* projectPoints method
These methods use `double` precision operations. Because the ESP32 only have a single precision Floating Point Unit (FPU), the operations are very slow. (TODO: have modified the methods to use only `float`, but must now add them to this repo).
Parallel support
-----------------------
TODO

64
esp32/doc/detailed_build_procedure.md
View File

@ -1,6 +1,6 @@
# Detailed build procedure
The last way explains all the commands and modifications done to be able to compile and run OpenCV on the ESP32.
This doc details all the commands and modifications done to be able to compile and run OpenCV on the ESP32.
@ -176,61 +176,6 @@ When the compilation has ended, the libs are in the `build/lib` folder.
## Compiling esp-idf project using OpenCV:
When the OpenCV library is cross-compiled, we have in result `*.a` files located in `build/lib` folder. We now want to try to compile an example project using OpenCV on the esp32. A basic example of esp-idf project can be found in [esp32/examples/esp_opencv_basic](esp32/examples/esp_opencv_basic). This project simply creates an OpenCV matrix, fill it with values and prints it on the console.
Esp-idf environment uses cmake and is separated in components. Because OpenCV libs were compiled outside this example project, we use the pre-built library functionality of esp-idf (https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/build-system.html#using-prebuilt-libraries-with-components).
Here are the things done to add the OpenCV library to the project:
* Create a folder named `opencv/` into the `main/` component's folder
* Copy the generated libraries (`libade.a`, `libopencv_core.a`, `libopencv_imgproc.a` and `libopencv_imgcodecs.a`) into this directory
* Create the folder `opencv2/` into this directory, and copy into it the needed headers files :
* `cvconfig.h`
* `opencv_modules.hpp`
* `opencv.hpp`
* `core.hpp`
* `imgproc.hpp`
* `imgcodecs.hpp`
* `core/` folder
* `imgproc/` folder
* `imgcodecs/` folder
* Link the libraries to the project by modifying the `CMakeList.txt` of the `main` project's component is as below :
```cmake
idf_component_register(
SRC main.cpp
INCLUDE_DIRS ./opencv
)
# Be aware that the order of the librairies is important
add_prebuilt_library(opencv_imgcodecs "opencv/libopencv_imgcodecs.a")
add_prebuilt_library(libpng "opencv/3rdparty/liblibpng.a")
add_prebuilt_library(libzlib "opencv/3rdparty/libzlib.a")
add_prebuilt_library(opencv_imgproc "opencv/libopencv_imgproc.a")
add_prebuilt_library(opencv_core "opencv/libopencv_core.a")
add_prebuilt_library(ade "opencv/libade.a")
target_link_libraries(${COMPONENT_LIB} PRIVATE opencv_imgcodecs)
target_link_libraries(${COMPONENT_LIB} PRIVATE libpng)
target_link_libraries(${COMPONENT_LIB} PRIVATE libzlib)
target_link_libraries(${COMPONENT_LIB} PRIVATE opencv_imgproc)
target_link_libraries(${COMPONENT_LIB} PRIVATE opencv_core)
target_link_libraries(${COMPONENT_LIB} PRIVATE ade)
```
* Finally, include the OpenCV headers needed into your source files:
```c++
#include "opencv2/core.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
```
## Modified files for OpenCV to compile and run:
To get the cmake configuration and make compilation to work, some modifications on OpenCV files had to be done. They are listed below.
@ -418,3 +363,10 @@ When the `make` command compiles successfully, the library was tested with an ex
* Disabling the Bluetooth (~56kB) and Trace Memory in the menuconfig
* Disabling some OpenCV features not mandatory and taking lot of memory. To find variables taking too much space, the `build/<project-name>.map` file is useful (looking for big variables under the `.dram0.bss` section).
* `imgproc/color_lab.cpp` has variables (trilinearLUTE, InvGammaTab and LabCbrtTab) taking ~88 kB. They are used for colorspace conversions in and from CIE LAB and XYZ images. These functions were removed.
* TODO: There are maybe other ways to save space in this segment:
* Allow .bss segment placed in external memory
* This option of the menuconfig uses external RAM to store zero-initialized data from the lwIP, net80211, libpp and bluedroid libraries
* Additional data can be moved from the internal BSS segment to external RAM by applying the macro `EXT_RAM_ATTR` to any static declaration (which is not initialized to a non-zero value).
* TODO: See what needs to be included and if it's possible to use this esp macro in OpenCV