Update Android OpenCL sample #24715
Update Android OpenCL sample and tutorial text.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
@ -6,21 +6,19 @@ Use OpenCL in Android camera preview based CV application {#tutorial_android_ocl
| | |
| | |
| -: | :- |
| -: | :- |
| Original author | Andrey Pavlenko |
| Original author | Andrey Pavlenko, Alexander Panov |
| Compatibility | OpenCV >= 3.0 |
| Compatibility | OpenCV >= 4.9 |
@warning
This tutorial is deprecated.
This guide was designed to help you in use of [OpenCL ™](https://www.khronos.org/opencl/) in Android camera preview based CV application.
This guide was designed to help you in use of [OpenCL ™](https://www.khronos.org/opencl/) in Android camera preview based CV application.
It was written for [Eclipse-based ADT tools](http://developer.android.com/tools/help/adt.html)
Tutorial was written for [Android Studio](http://developer.android.com/tools/studio/index.html) 2022.2.1. It was tested with Ubuntu 22.04.
(deprecated by Google now), but it easily can be reproduced with [Android Studio](http://developer.android.com/tools/studio/index.html).
This tutorial assumes you have the following installed and configured:
This tutorial assumes you have the following installed and configured:
- JDK
- Android Studio (2022.2.1.+)
- Android SDK and NDK
- JDK 17
- Eclipse IDE with ADT and CDT plugins
- Android SDK
- Android NDK (25.2.9519653+)
- download OpenCV source code from [github](git@github.com:opencv/opencv.git) or from [releases](https://opencv.org/releases/) and build by [instruction on wiki](https://github.com/opencv/opencv/wiki/Custom-OpenCV-Android-SDK-and-AAR-package-build).
It also assumes that you are familiar with Android Java and JNI programming basics.
It also assumes that you are familiar with Android Java and JNI programming basics.
If you need help with anything of the above, you may refer to our @ref tutorial_android_dev_intro guide.
If you need help with anything of the above, you may refer to our @ref tutorial_android_dev_intro guide.
@ -30,6 +28,56 @@ This tutorial also assumes you have an Android operated device with OpenCL enabl
The related source code is located within OpenCV samples at
The related source code is located within OpenCV samples at
libOpenCL.so may be provided with BSP or just downloaded from any OpenCL-cabaple Android device with relevant arhitecture.
@code{.bash}
cd your_path/ANDROID_OPENCL_SDK && mkdir lib && cd lib
adb pull /system/vendor/lib64/libOpenCL.so
@endcode
System verison of libOpenCL.so may have a lot of platform specific dependencies. `-Wl,--allow-shlib-undefined` flag allows
to ignore 3rdparty symbols if they are not used during the build.
The following CMake line allows to link the JNI part against standard OpenCL, but not include the loadLibrary into
application package. System OpenCL API is used in run-time.
@code
target_link_libraries(${target} -lOpenCL)
@endcode
2. __Build custom OpenCV Android SDK with OpenCL.__
OpenCL support (T-API) is disabled in OpenCV builds for Android OS by default.
but it's possible to rebuild locally OpenCV for Android with OpenCL/T-API enabled: use `-DWITH_OPENCL=ON` option for CMake.
You also need to specify the path to the Android OpenCL SDK: use `-DANDROID_OPENCL_SDK=path_to_your_Android_OpenCL_SDK` option for CMake.
If you are building OpenCV using `build_sdk.py` please follow [instruction on wiki](https://github.com/opencv/opencv/wiki/Custom-OpenCV-Android-SDK-and-AAR-package-build).
Set these CMake parameters in your `.config.py`, e.g. `ndk-18-api-level-21.config.py`:
public void surfaceCreated(SurfaceHolder holder) {
super.surfaceCreated(holder);
}
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
super.surfaceDestroyed(holder);
}
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
super.surfaceChanged(holder, format, w, h);
}
@Override
public void onResume() {
super.onResume();
mRenderer.onResume();
}
@Override
public void onPause() {
mRenderer.onPause();
super.onPause();
}
}
@endcode
__Note__: we use two renderer classes: one for legacy [Camera](http://developer.android.com/reference/android/hardware/Camera.html) API
@note we use two renderer classes: one for legacy [Camera](http://developer.android.com/reference/android/hardware/Camera.html) API
and another for modern [Camera2](http://developer.android.com/reference/android/hardware/camera2/package-summary.html).
and another for modern [Camera2](http://developer.android.com/reference/android/hardware/camera2/package-summary.html).
A minimal `Renderer` class can be implemented in Java (OpenGL ES 2.0 [available](http://developer.android.com/reference/android/opengl/GLES20.html) in Java),
A minimal `Renderer` class can be implemented in Java (OpenGL ES 2.0 [available](http://developer.android.com/reference/android/opengl/GLES20.html) in Java),
but since we are going to modify the preview texture with OpenCL let's move OpenGL stuff to JNI.
but since we are going to modify the preview texture with OpenCL let's move OpenGL stuff to JNI.
But instead of writing OpenCL code by yourselves you may want to use __OpenCV T-API__ that calls OpenCL implicitly.
But instead of writing OpenCL code by yourselves you may want to use __OpenCV T-API__ that calls OpenCL implicitly.
All that you need is to pass the created OpenCL context to OpenCV (via `cv::ocl::attachContext()`) and somehow wrap OpenGL texture with `cv::UMat`.
All that you need is to pass the created OpenCL context to OpenCV (via `cv::ocl::attachContext()`) and somehow wrap OpenGL texture with `cv::UMat`.
Unfortunately `UMat` keeps OpenCL _buffer_ internally, that can't be wrapped over either OpenGL _texture_ or OpenCL _image_ - so we have to copy image data here:
Unfortunately `UMat` keeps OpenCL _buffer_ internally, that can't be wrapped over either OpenGL _texture_ or OpenCL _image_ - so we have to copy image data here:
- @note By default the OpenCL support (T-API) is disabled in OpenCV builds for Android OS (so it's absent in official packages as of version 3.0),
but it's possible to rebuild locally OpenCV for Android with OpenCL/T-API enabled: use `-DWITH_OPENCL=YES` option for CMake.
@note We have to make one more image data copy when placing back the modified image to the original OpenGL texture via OpenCL image wrapper.
@code{.cmd}
cd opencv-build-android
path/to/cmake.exe -GNinja -DCMAKE_MAKE_PROGRAM="path/to/ninja.exe" -DCMAKE_TOOLCHAIN_FILE=path/to/opencv/platforms/android/android.toolchain.cmake -DANDROID_ABI="armeabi-v7a with NEON" -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON path/to/opencv
path/to/ninja.exe install/strip
@endcode
To use your own modified `libopencv_java4.so` you have to keep inside your APK, not to use OpenCV Manager and load it manually via `System.loadLibrary("opencv_java4")`.
Alexander Panov
11 months ago
committed by
GitHub