使用机器学习的应用程序通常需要高计算能力。这些计算通常发生在显卡的GPU上。Raspberry Pi并不专门设计用于运行计算密集型应用程序。但Google Coral USB加速器能在此提供帮助!借助此设备,我们可以在视频中实现实时计算,如对象识别。
Raspberry Pi*
Edge TPU:Google Coral USB加速器
官方Raspberry Pi摄像头模块 或 USB网络摄像头
用于识别的简单对象(办公用品、水果等)
最佳选择:一个为Raspberry Pi和USB加速器配备散热器的外壳(也可3D打印)。
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" |
sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
-
sudo apt-get update
sudo apt-get install libedgetpu1-std
sudo apt-get install python3-pycoral --yes
pip3 show tflite_runtime
Name: tflite-runtime
Version: 2.5.0
Summary: TensorFlow Lite is for mobile and embedded devices.
Home-page: https://www.tensorflow.org/lite/
Author: Google, LLC
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /usr/lib/python3/dist-packages
Requires:
Required-by: pycoral
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" |
sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
-
sudo apt-get update
sudo apt-get install python3-tflite-runtime
mkdir google-coral && cd google-coral
git clone https://github.com/google-coral/examples-camera --depth 1
cd examples-camera
sh download_models.sh
cd opencv
bash install_requirements.sh
python3 detect.py
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""A demo that runs object detection on camera frames using OpenCV.
TEST_DATA=../all_models
Run face detection model:
python3 detect.py \
--model ${TEST_DATA}/mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite
Run coco model:
python3 detect.py \
--model ${TEST_DATA}/mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite \
--labels ${TEST_DATA}/coco_labels.txt
"""
import argparse
import cv2
import os
from pycoral.adapters.common import input_size
from pycoral.adapters.detect import get_objects
from pycoral.utils.dataset import read_label_file
from pycoral.utils.edgetpu import make_interpreter
from pycoral.utils.edgetpu import run_inference
def main():
default_model_dir = '../all_models'
default_model = 'mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite'
default_labels = 'coco_labels.txt'
parser = argparse.ArgumentParser()
parser.add_argument('--model', help='.tflite model path',
default=os.path.join(default_model_dir,default_model))
parser.add_argument('--labels', help='label file path',
default=os.path.join(default_model_dir, default_labels))
parser.add_argument('--top_k', type=int, default=3,
help='number of categories with highest score to display')
parser.add_argument('--camera_idx', type=int, help='Index of which video source to use. ', default = 0)
parser.add_argument('--threshold', type=float, default=0.1,
help='classifier score threshold')
args = parser.parse_args()
print('Loading {} with {} labels.'.format(args.model, args.labels))
interpreter = make_interpreter(args.model)
interpreter.allocate_tensors()
labels = read_label_file(args.labels)
inference_size = input_size(interpreter)
cap = cv2.VideoCapture(args.camera_idx)
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
cv2_im = frame
cv2_im_rgb = cv2.cvtColor(cv2_im, cv2.COLOR_BGR2RGB)
cv2_im_rgb = cv2.resize(cv2_im_rgb, inference_size)
run_inference(interpreter, cv2_im_rgb.tobytes())
objs = get_objects(interpreter, args.threshold)[:args.top_k]
cv2_im = append_objs_to_img(cv2_im, inference_size, objs, labels)
cv2.imshow('frame', cv2_im)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
def append_objs_to_img(cv2_im, inference_size, objs, labels):
height, width, channels = cv2_im.shape
scale_x, scale_y = width / inference_size[0], height / inference_size[1]
for obj in objs:
bbox = obj.bbox.scale(scale_x, scale_y)
x0, y0 = int(bbox.xmin), int(bbox.ymin)
x1, y1 = int(bbox.xmax), int(bbox.ymax)
percent = int(100 * obj.score)
label = '{}% {}'.format(percent, labels.get(obj.id, obj.id))
cv2_im = cv2.rectangle(cv2_im, (x0, y0), (x1, y1), (0, 255, 0), 2)
cv2_im = cv2.putText(cv2_im, label, (x0, y0+30),
cv2.FONT_HERSHEY_SIMPLEX, 1.0, (255, 0, 0), 2)
return cv2_im
if __name__ == '__main__':
main()
首先,包含了所需的PyCoral库。 在主函数中,定义了可以从命令行传递的参数(如指定模型等)。 加载模型及其标签,并根据模型确定尺寸(此处为300×300)。 然后打开视频流(cap = cv2.VideoCapture(args.camera_idx))。 接下来是核心部分:
objs = get_objects(interpreter, args.threshold)[:args.top_k]
此过程中会确定“分类分数”最高的3个元素(且分数需高于阈值)。 随后,在图像上标记每个检测到的物体。
[
Object(id=16, score=0.5, bbox=BBox(xmin=-2, ymin=102, xmax=158, ymax=296)),
Object(id=0, score=0.16015625, bbox=BBox(xmin=6, ymin=114, xmax=270, ymax=300)),
Object(id=61, score=0.12109375, bbox=BBox(xmin=245, ymin=166, xmax=301, ymax=302))
]
{0: 'person', 1: 'bicycle', 2: 'car', 3: 'motorcycle', 4: 'airplane', 5: 'bus',
6: 'train', 7: 'truck', 8: 'boat', 9: 'traffic light', 10: 'fire hydrant', 1
2: 'stop sign', 13: 'parking meter', 14: 'bench', 15: 'bird', 16: 'cat', 17: 'dog',
18: 'horse', 19: 'sheep', 20: 'cow', 21: 'elephant', 22: 'bear', 23: 'zebra',
24: 'giraffe', 26: 'backpack', 27: 'umbrella', 30: 'handbag', 31: 'tie',
32: 'suitcase', 33: 'frisbee', 34: 'skis', 35: 'snowboard', 36: 'sports ball',
37: 'kite', 38: 'baseball bat', 39: 'baseball glove', 40: 'skateboard',
41: 'surfboard', 42: 'tennis racket', 43: 'bottle', 45: 'wine glass', 46: 'cup',
47: 'fork', 48: 'knife', 49: 'spoon', 50: 'bowl', 51: 'banana', 52: 'apple',
53: 'sandwich', 54: 'orange', 55: 'broccoli', 56: 'carrot', 57: 'hot dog',
58: 'pizza', 59: 'donut', 60: 'cake', 61: 'chair', 62: 'couch', 63: 'potted plant',
64: 'bed', 66: 'dining table', 69: 'toilet', 71: 'tv', 72: 'laptop', 73: 'mouse',
74: 'remote', 75: 'keyboard', 76: 'cell phone', 77: 'microwave', 78: 'oven',
79: 'toaster', 80: 'sink', 81: 'refrigerator', 83: 'book', 84: 'clock', 85: 'vase',
86: 'scissors', 87: 'teddy bear', 88: 'hair drier', 89: 'toothbrush'}
found_scores = [o.score for o in objs if o.id == 5]
if len(found_scores) > 0 and max(found_scores) >= 0.8:
想咨询树莓派标品和树莓派工业产品的小伙伴,欢迎联系我们~
我们会定期更新~
关注上海晶珩~
了解更多树莓派相关内容~