• 10
name

A PHP Error was encountered

Severity: Notice

Message: Undefined index: userid

Filename: views/question.php

Line Number: 191

Backtrace:

File: /home/prodcxja/public_html/questions/application/views/question.php
Line: 191
Function: _error_handler

File: /home/prodcxja/public_html/questions/application/controllers/Questions.php
Line: 433
Function: view

File: /home/prodcxja/public_html/questions/index.php
Line: 315
Function: require_once

I'm working on and android app that can digitize numbers from a paper. I use native OpenCV code to find the numbers on the image. After that I want to use OpenCV's dnn module to recognise the number. A nice tutorial on creating the neural net can be found here:
https://www.youtube.com/watch?v=kFWKdLOxykE
The mnist_convnet_graph.pbtxt begins with this:

node {
  name: "conv2d_1_input"
  op: "Placeholder"
  attr {
    key: "dtype"
    value {
    type: DT_FLOAT
  }
  }
  attr {
    key: "shape"
    value {
      shape {
        dim {
          size: -1
        }
        dim {
          size: 28
        }
        dim {
          size: 28
        }
        dim {
          size: 1
        }
      }
    }
  }
}

So the input is a 28x28 grayscale image.
In the tutorial java code is used to use the neural net. However, I would like to use it in C++, because of the speed. I successfully load the model with cv::dnn::Net Dnn.readNetFromTensorflow(String model, String config); and pass the object to the NDK side. I create the input for the neural net with the following:

// The part of the image, we are interested in.
Rect roi(static_cast<int>(w), static_cast<int>(h),
             static_cast<int>(w), static_cast<int>(h));
Mat cropped(image_gray, roi);
// Resize image to 28x28.
Mat resized;
cv::resize(cropped, resized, Size(28,28));

After that, the forwarding should work:

const double IN_SCALE_FACTOR = 0.003921; // 1.0/255.0
Mat blob = dnn::blobFromImage(resized, IN_SCALE_FACTOR, Size(28,28));
net.setInput(blob);
Mat detections = net.forward();

where net is the passed cv::dnn::Net object. But the net.forward() command fails and gives:
OpenCV(3.4.5) Error: Assertion failed (inputs.size() == requiredOutputs) in virtual bool cv::dnn::experimental_dnn_34_v11::DataLayer::getMemoryShapes(const std::vector >&, int, std::vector >&, std::vector >&) const, file /build/3_4_pack-android/opencv/modules/dnn/src/dnn.cpp, line 681

I also tried:

  • cropping the rgb image
  • Mat blob = dnn::blobFromImage(resized, 1.0f, Size(28,28));
  • Not using blobFromImage, but net.setInput(resized); instead

but none of these led to the solutions. Anyone has a solutions for this? Any suggestion or idea will be appreciated.

      • 1
    • Sure. It is a 1x1x28x28 image. I don't know if it's appropriate for the input. Because in the mnist_convnet_graph.pbtxt file the order is not the same.
      • 2
    • 1x1x28x28 is OK. It's just different data layout which OpenCV can manage. Could you please share an entire text .pbtxt file to see it? And please specify version of OpenCV you use. BTW, have you tried to import model using just a binary .pb file?
      • 1
    • Sure. It's here: link. I use OpenCV version 3.4.5 and Android Studio 3.3. The thing is that the readNetFromTensorflow() function's first argument is the .pb file (~17,4 MB) and the second is the .pbtxt (~284 KB).
    • hmm, it looks like the model is serialized in training mode (it should be in testing mode). There is a procedure called freezing which disabled all the training only nodes (there are in your graph).

Okay, I managed to solve my problem.

Firstly, I realized that the .pb and .pbtxt files are in the wrong directory and getting 2 Failed to upload a file information log.

After putting the files into the right directory, I faced with problem:
error: (-215:Assertion failed) const_layers.insert(std::make_pair(name, li)).second in function 'void cv::dnn::experimental_dnn_34_v11::{anonymous}::addConstNodes(opencv_tensorflow::GraphDef&, std::map<cv::String, int>&, std::set<cv::String>&)'

As Dmitry Kurtaev suggested here, I removed the .pbtxt from the Dnn.readNetFromTensorflow. After that I got error:
OpenCV(3.4.5) Error: Unspecified error (Can't create layer "flatten_1/Shape" of type "Shape") in cv::Ptr<cv::dnn::experimental_dnn_34_v11::Layer> cv::dnn::experimental_dnn_34_v11::LayerData::getLayerInstance(), file /build/3_4_pack-android/opencv/modules/dnn/src/dnn.cpp, line 513

This led me to a link, which I found in Dmitry Kurtaev's comment here. After doing the suggested modifications (removing Const nodes, modifying and removing flatten nodes) on the .pbtxt file, finally I got no errors and got successfully running the neural net.

Note: Adding K.backend.set_learning_phase(0) before creating the model, also can be useful.

  • 1
Reply Report