OpenCv在计算机视觉方面大有用处，这篇文章主要记录在Android Framework集成OpenCv，对Camera数据流进行处理，遇到的问题记录备忘。
openc源码下载地址：
https://opencv.org/releases/
一、OpencV在framework层集成
（下面的三方算法，是指的三方算法采用到了opencv）
1、三方算法是源码
(如下面所示，image.cpp是三方算法源码，该源码image.cpp文件中调用到opencv)
// -----Android.bpcc_library_shared {name: "libimage",rtti:true,srcs: ["image/image.cpp",],include_dirs: ["frameworks/av/services/camera/libcameraservice/image/include",],shared_libs: ["libopencv_java4","liblog",],export_include_dirs: ["."],cflags: ["-Wall","-Wextra","-Werror","-Wno-ignored-qualifiers","-fPIC","-ferror-limit=0",],cppflags: ["-fno-rtti","-fexceptions",],}
2、三方算法是so库
1 ） 如下所示，libimage是三方算法提供的so库，我们自己封装了个libProcess的so，libProcess中调用三方算法的libimage,三方的libimage算法库，会调用到opencv；
//----- Android.bp内容cc_prebuilt_library_shared {name: "libopencv_java4",arch: {arm: {srcs: ["image/lib/armeabi-v7a/libopencv_java4.so"],},arm64: {srcs: ["image/lib/arm64-v8a/libopencv_java4.so"],}},shared_libs: ["libandroid", "libc++", "liblog", "libmediandk", "libjnigraphics","libz","libdl","libm"],check_elf_files: false,}cc_prebuilt_library_shared {name: "libimage",arch: {arm: {srcs: ["./image/lib/armeabi-v7a/libimage.so"],},arm64: {srcs: ["./image/lib/arm64-v8a/libimage.so"],}},shared_libs: ["libopencv_java4", "libandroid", "libc++", "liblog", "libmediandk", "libjnigraphics","libz","libdl","libm"],check_elf_files: false,}cc_library_shared {name: "libProcess",//编译引用到opencv相关头文件，需要设为true，否则编译报错rtti:true,srcs: ["image/ImageProcess.cpp",],//调用到的opencv,opencv相关的头文件路径include_dirs: ["frameworks/av/services/camera/libcameraservice/image/include",],shared_libs: ["libimage","libopencv_java4","liblog",],export_include_dirs: ["."],cflags: ["-Wall","-Wextra","-Werror","-Wno-ignored-qualifiers","-fPIC","-ferror-limit=0",],cppflags: ["-fno-rtti","-fexceptions",],}
2 ） 正常来说，我们调用算法的so库，其实是不需要再单独封装一个so库。这个封装成一个so库的原因是，三方算法调用到了opencv，opencv头文件的编译，需要 rtti 设为true，而rtti设为true,会影响到cameraservice里面其它代码的编译。
所以比较好的解决方法就是，单独封装一个调用三方算法库的so，在这个so里面，rtti 设为true，这样不影响其它代码的编译。
----- 单独封装的调用三方算法库(libimage) 的 ImageProcess.cpp文件内容：#include <iostream>#include <opencv2/opencv.hpp>#include <fstream>#include <ctime>#include<cmath>//三方算法库头文件#include "process.h"using namespace cv;using namespace std;void imageProcess(uint8_t* buffer, uint width, uint height){//process是三方算法库的接口process(buffer, width, height);}
在framework层获取camera相关数据，可以查看下面代码。
在下面代码逻辑里面，可以拿到拍照的jpeg以及录像的yuv数据。
frameworks/av/services/camera/libcameraservice/device3/Camera3OutputStream.cpp
Camera3OutputStream::returnBufferCheckedLocked{if (getFormat() == HAL_PIXEL_FORMAT_BLOB && getDataSpace() == HAL_DATASPACE_V0_JFIF && mImageDumpMask) {dumpImageToDisk(timestamp, anwBuffer, anwReleaseFence);}ALOGE("%s:getFormat():%d ,width:%d ,HAL_PIXEL_FORMAT_YCbCr_420_888:%d ", __FUNCTION__,getFormat(),buffer.stream->width,HAL_PIXEL_FORMAT_YCbCr_420_888);//新添加内容，判断到是yuv format,才走下面的逻辑if (getFormat() == 34) {processYuv(anwBuffer,buffer.stream->width, buffer.stream->height,anwReleaseFence);}}void Camera3OutputStream::processYuv(ANativeWindowBuffer* anwBuffer, int width,int height,int fence) {ALOGE("%s: width: %d,height: %d,usage: %d", __FUNCTION__, width, height, usage);// Lock the image for CPU readsp<GraphicBuffer> graphicBuffer = GraphicBuffer::from(anwBuffer);void* mapped = nullptr;base::unique_fd fenceFd(dup(fence));status_t res = graphicBuffer->lockAsync(GraphicBuffer::USAGE_SW_READ_OFTEN, &mapped,fenceFd.get());if (res != OK) {ALOGE("%s: Failed to lock the buffer: %s (%d)", __FUNCTION__, strerror(-res), res);return;}//这个地方，mapped拿到的就是yuv数据,就可以把mapped送给算法进行处理.processImage((uint8_t*)mapped,width,height);//也可以把yuv数据保存到本地进行debug.}
二、opencv在Android源码集成遇到的问题汇总
1、not specified in shared_libs
【action】 这类错误，就是我们的目标文件，需要引用到的一些so库，但是我们的bp或者mk文件没有申明。
如下图所示，错误信息里面，也有Fix suggestion，根据提示来做修改就好。
DT_NEEDED "liblog.so" is not specified in shared_libsFix suggestions:Android.bp: shared_libs: ["libc++_shared", "libc", "libdl", "libjnigraphics", "liblog", "libm", "libmediandk", "libz"],Android.mk: LOCAL_SHARED_LIBRARIES := libc++_shared libc libdl libjnigraphics liblog libm libmediandk libzIf the fix above doesn't work, bypass this check with:Android.bp: check_elf_files: false,Android.mk: LOCAL_CHECK_ELF_FILES := false
2、(native:vendor) can not link against libc++_shared (native:platform)
【action】问题原因是，android8.0之后，system和vendor库进行了区分，除了一些特殊的so库，很多so库，比vendor模块的代码，不能去访问到system下的so库。vndk ?
vendor/mediatek/proprietary/hardware/mtkcam3/***/Android.mk: error: "libopencv_java4 (native:vendor) can not link against libc++_shared (native:platform)"类似上面这种报错，如果是vendor下去访问system下的so，我们可以找到system下该so源码的Android.bp文件，添加vendor_available: true 来进行尝试。
diff --git a/frameworks/base/native/graphics/jni/Android.bp b/frameworks/base/native/graphics/jni/Android.bpindex 1709dfd973d..22432812430 100644--- a/frameworks/base/native/graphics/jni/Android.bp+++ b/frameworks/base/native/graphics/jni/Android.bp@@ -47,6 +47,9 @@ cc_library_shared {static_libs: ["libarect"],++ proprietary:true,+
For Android.bp:vendor_available: true
For Android.mk:LOCAL_VENDOR_MODULE := true
如果是涉及到多个so的嵌套使用，就会比较麻烦，如下面的，libjnigraphics里面，会用到libhwui，而libhwui也是不允许vendor去访问的，libhwui里面，又会用到其它的vendor无法访问的so。类似这种情况？关闭vndk？
网友提供的一些方案（https://blog.csdn.net/qq_35345103/article/details/110390454），试了没能解决。
FAILED: out/soong/.intermediates/frameworks/base/native/graphics/jni/libjnigraphics/android_vendor.31_arm64_armv8-2a_cortex-a55_shared/libjnigraphics.soecho "module libjnigraphics missing dependencies: libhwui_internal_headers{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:}, libhwui{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:,link:shared}, libandroid{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:,link:shared}" && falsemodule libjnigraphics missing dependencies: libhwui_internal_headers{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:}, libhwui{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:,link:shared}, libandroid{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:,link:shared}
vndk相关内容，可以查看官网介绍：
https://source.android.google.cn/docs/core/architecture/vndk/build-system?hl=ca
3、error: cannot use 'throw' with exceptions disabled
frameworks/av/services/camera/libcameraservice/***/include/opencv2/flann/kmeans_index.h:445:13: error: cannot use 'throw' with exceptions disabled FLANN_THROW(cv::Error::StsError, "Branching factor must be at least 2"); ^frameworks/av/services/camera/libcameraservice/image/include/opencv2/flann/general.h:55:32: note: expanded from macro 'FLANN_THROW'#define FLANN_THROW(TYPE, STR) throw FLANNException(STR) ^fatal error: too many errors emitted, stopping now [-ferror-limit=]
【action】
Android.bp中添加如下内容可以解决：
cppflags: [ "-fexceptions", ],
Android.mk的修改
LOCAL_CPPFLAGS += -fexceptions
4、 error: use of ***requires -frtti
frameworks/av/services/camera/libcameraservice/image/include/opencv2/flann/any.h:334:34: error: use of typeid requires -frtti return policy->type() == typeid(T);
【action】
android.bp中添加如下内容可以解决：rtti: true,(如果是cppflags中添加如下内容，会继续报该错误)cppflags: ["-frtti",],
5、vndk能否关闭掉，怎么关闭？
《Android Camera开发入门》、《Camx初认识》已经上架，可以点击了解 -> 小驰成长圈 ｜期待见证彼此的成长
觉得不错，点个赞呗
