diff --git a/examples/idenprof/idenprof.json b/examples/idenprof/idenprof.json new file mode 100644 index 00000000..be683edc --- /dev/null +++ b/examples/idenprof/idenprof.json @@ -0,0 +1,12 @@ +{ + "0" : "chef", + "1" : "doctor", + "2" : "engineer", + "3" : "farmer", + "4" : "firefighter", + "5" : "judge", + "6" : "mechanic", + "7" : "pilot", + "8" : "police", + "9" : "waiter" +} \ No newline at end of file diff --git a/examples/idenprof/model_resnet50_ex-017_acc-0.787_vacc0.747.h5 b/examples/idenprof/model_resnet50_ex-017_acc-0.787_vacc0.747.h5 new file mode 100644 index 00000000..52c47810 Binary files /dev/null and b/examples/idenprof/model_resnet50_ex-017_acc-0.787_vacc0.747.h5 differ diff --git a/imageai/Classification/README.md b/imageai/Classification/Classification Readme.md similarity index 56% rename from imageai/Classification/README.md rename to imageai/Classification/Classification Readme.md index bb4def1f..0b87d065 100644 --- a/imageai/Classification/README.md +++ b/imageai/Classification/Classification Readme.md @@ -12,22 +12,21 @@ A **DeepQuest AI** project [https://deepquestai.com](https://deepquestai.com) ImageAI provides 4 different algorithms and model types to perform image prediction. To perform image prediction on any picture, take the following simple steps. The 4 algorithms provided for - image prediction include **MobileNetV2**, **ResNet50**, **InceptionV3** and **DenseNet121**. Each of these - algorithms have individual model files which you must use depending on the choice of your algorithm. To download the - model file for your choice of algorithm, click on any of the links below: +image prediction include **MobileNetV2**, **ResNet50**, **InceptionV3** and **DenseNet121**. Each of these +algorithms have individual model files which you must use depending on the choice of your algorithm. -- **[MobileNetV2](https://github.com/OlafenwaMoses/ImageAI/releases/download/essentials-v5/mobilenet_v2.h5)** _(Size = 4.82 mb, fastest prediction time and moderate accuracy)_ -- **[ResNet50](https://github.com/OlafenwaMoses/ImageAI/releases/download/essentials-v5/resnet50_imagenet_tf.2.0.h5)** by Microsoft Research _(Size = 98 mb, fast prediction time and high accuracy)_ - - **[InceptionV3](https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/inception_v3_weights_tf_dim_ordering_tf_kernels.h5)** by Google Brain team _(Size = 91.6 mb, slow prediction time and higher accuracy)_ - - **[DenseNet121](https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/DenseNet-BC-121-32.h5)** by Facebook AI Research _(Size = 31.6 mb, slower prediction time and highest accuracy)_ - - Great! Once you have downloaded this model file, start a new python project, and then copy the model file to your project - folder where your python files (.py files) will be . Download the image below, or take any image on your computer - and copy it to your python project's folder. Then create a python file and give it a name; an example is `FirstPrediction.py`. - Then write the code below into the python file: +- **[MobileNetV2]** _(Fastest prediction time and moderate accuracy)_ +- **[ResNet50]** by Microsoft Research _(Fast prediction time and high accuracy)_ +- **[InceptionV3]** by Google Brain team _(Slow prediction time and higher accuracy)_ +- **[DenseNet121]** by Facebook AI Research _(Slower prediction time and highest accuracy)_ + +Great! Once you have chosen your model, start a new python project, and then copy the model file to your project +folder where your python files (.py files) will be. Download the image below, or take any image on your computer +and copy it to your python project's folder. Then create a python file and give it a name; an example is `FirstPrediction.py`. +Then write the code below into the python file: ### FirstPrediction.py -
+
```python from imageai.Classification import ImageClassification @@ -36,11 +35,10 @@ import os execution_path = os.getcwd() prediction = ImageClassification() -prediction.setModelTypeAsResNet50() -prediction.setModelPath(os.path.join(execution_path, "resnet50_imagenet_tf.2.0.h5")) -prediction.loadModel() +prediction.set_model_type('ResNet50') +prediction.load_model() -predictions, probabilities = prediction.classifyImage(os.path.join(execution_path, "1.jpg"), result_count=5 ) +predictions, probabilities = prediction.classify_image(os.path.join(execution_path, "1.jpg"), result_count=5) for eachPrediction, eachProbability in zip(predictions, probabilities): print(eachPrediction , " : " , eachProbability) ``` @@ -49,11 +47,11 @@ Sample Result: ![](../../data-images/1.jpg) ``` -convertible : 52.459555864334106 -sports_car : 37.61284649372101 -pickup : 3.1751200556755066 -car_wheel : 1.817505806684494 -minivan : 1.7487050965428352 +convertible : 59.43046808242798 +sports_car : 22.52226620912552 +minivan : 5.942000821232796 +pickup : 5.5309634655714035 +car_wheel : 1.2574399821460247 ``` The code above works as follows: @@ -69,56 +67,42 @@ The above line obtains the path to the folder that contains your python file (in ```python prediction = ImageClassification() -prediction.setModelTypeAsResNet50() -prediction.setModelPath(os.path.join(execution_path, "resnet50_imagenet_tf.2.0.h5")) +prediction.set_model_type('ResNet50') ``` -In the lines above, we created and instance of the `ImagePrediction()` class in the first line, then we set the model type of the prediction object to ResNet by caling the `.setModelTypeAsResNet50()` in the second line and then we set the model path of the prediction object to the path of the model file (`resnet50_imagenet_tf.2.0.h5`) we copied to the python file folder in the third line. +In the lines above, we created and instance of the `ImagePrediction()` class in the first line, then set the model type of the prediction object to ResNet by caling the `.set_model_type('ResNet50')` in the second line. ```python -predictions, probabilities = prediction.classifyImage(os.path.join(execution_path, "1.jpg"), result_count=5 ) +predictions, probabilities = prediction.classify_image(os.path.join(execution_path, "1.jpg"), result_count=5 ) ``` -In the above line, we defined 2 variables to be equal to the function called to predict an image, which is the `.classifyImage()` function, into which we parsed the path to our image and also state the number of prediction results we want to have (values from 1 to 1000) parsing `result_count=5`. The `.classifyImage()` function will return 2 array objects with the first (**predictions**) being an array of predictions and the second (**percentage_probabilities**) being an array of the corresponding percentage probability for each prediction. +In the above line, we defined 2 variables to be equal to the function called to predict an image, which is the `.classify_image()` function, into which we parsed the path to our image and also state the number of prediction results we want to have (values from 1 to 1000) parsing `result_count=5`. The `.classify_image()` function will return 2 array objects with the first (**predictions**) being an array of predictions and the second (**probabilities**) being an array of the corresponding percentage probability for each prediction. ```python for eachPrediction, eachProbability in zip(predictions, probabilities): - print(eachPrediction, " : " , eachProbability) + print(eachPrediction, ':' , eachProbability) ``` -The above line obtains each object in the **predictions** array, and also obtains the corresponding percentage probability from the **percentage_probabilities**, and finally prints the result of both to console. +The above line obtains each object in the **predictions** array, and also obtains the corresponding percentage probability from the **probabilities**, and finally prints the result of both to console. -### Prediction Speed + ### Image Input Types @@ -196,25 +180,24 @@ great_grey_owl : 0.0699841941241175 Previous version of **ImageAI** supported only file inputs and accepts file paths to an image for image prediction. Now, **ImageAI** supports 3 input types which are **file path to image file**(default), **numpy array of image** and **image file stream**. This means you can now perform image prediction in production applications such as on a web server and system - that returns file in any of the above stated formats. +that returns file in any of the above stated formats. To perform image prediction with numpy array or file stream input, you just need to state the input type -in the `.classifyImage()` function. See example below. +in the `.classify_image()` function. See example below. ```python -predictions, probabilities = prediction.classifyImage(image_array, result_count=5 , input_type="array" ) # For numpy array input type -predictions, probabilities = prediction.classifyImage(image_stream, result_count=5 , input_type="stream" ) # For file stream input type +predictions, probabilities = prediction.classify_image(image_array, result_count=5 , input_type="array" ) # For numpy array input type +predictions, probabilities = prediction.classify_image(image_stream, result_count=5 , input_type="stream" ) # For file stream input type ``` ### Prediction in MultiThreading
When developing programs that run heavy task on the deafult thread like User Interfaces (UI), - you should consider running your predictions in a new thread. When running image prediction using ImageAI in - a new thread, you must take note the following: -- You can create your prediction object, set its model type, set model path and json path -outside the new thread. -- The `.loadModel()` must be in the new thread and image prediction (`classifyImage()`) must take place in th new thread. +you should consider running your predictions in a new thread. When running image prediction using ImageAI in +a new thread, you must take note the following: +- You can create your prediction object, set its model type, set model path and json path outside the new thread. +- The `.load_model()` must be in the new thread and image prediction (`classify_image()`) must take place in th new thread. Take a look of a sample code below on image prediction using multithreading: ```python @@ -225,8 +208,7 @@ import threading execution_path = os.getcwd() prediction = ImageClassification() -prediction.setModelTypeAsResNet() -prediction.setModelPath( os.path.join(execution_path, "resnet50_imagenet_tf.2.0.h5")) +prediction.set_model_type('ResNet50') picturesfolder = os.environ["USERPROFILE"] + "\\Pictures\\" allfiles = os.listdir(picturesfolder) @@ -235,24 +217,24 @@ class PredictionThread(threading.Thread): def __init__(self): threading.Thread.__init__(self) def run(self): - prediction.loadModel() + prediction.load_model() for eachPicture in allfiles: if eachPicture.endswith(".png") or eachPicture.endswith(".jpg"): - predictions, percentage_probabilities = prediction.predictImage(picturesfolder + eachPicture, result_count=1) + predictions, percentage_probabilities = prediction.classify_image(picturesfolder + eachPicture, result_count=1) for prediction, percentage_probability in zip(predictions, probabilities): print(prediction , " : " , percentage_probability) -predictionThread = PredictionThread () +predictionThread = PredictionThread() predictionThread.start() ``` -### Documentation + diff --git a/imageai/Classification/Custom/Changes.md b/imageai/Classification/Custom/Changes.md new file mode 100644 index 00000000..ee14c34d --- /dev/null +++ b/imageai/Classification/Custom/Changes.md @@ -0,0 +1,44 @@ +### Changes in ImageAI V2.2.0 +
+ +ImageAI hasn't been updated since TensorFlow 2.4.0 and were on 2.9.1 now! It's definitely time for some updates and in ImageAI 2.2.0 there were some big changes that involve making using ImageAI as seamless as possible for new users who just want something that works without much configuration while still allowing configuration for users who want to dive deeper into image classification but still aren't ready to build their own models from the ground up or just want something that's easier to use and configure without having to completely re-write your program (I'm the latter and that's why I decided to rewrite it for myself and then wanted to share with everyone else) + +NOTE: The custom image classification has been updated quite a bit with changes mostly happening just to get everything working with current versions of tensorflow. With that said, tests were not re-run with the newest version of tensor flow so some of the specific timings of things most likely will not be accurate anymore (they'll usually be faster) but keep that in mind throughout this readme. + +**Now time for the changes!** + +ImageAI Custom Classification Version 2.2.0 Changes: + +Setting models: +- The four model types included with ImageAI are currently still the same as in v2.1.6 however to streamline the package code, make the end user process more simple, and future proof ImageAI, models will no longer be set with setModelTypeAsResNet50(), etc. Models will now be set using set_model_type() with the model you'd like to use ex: trainer.setModelTypeAs('ResNet50') +- On top of changing how the model type is set, setting a model type is no longer required. In almost every case that I've seen, people tend to use ResNet50 so that is now the default model that will be set if set_model_type() is not called. Note: This is mostly to make the experience easier for beginner users + +Model training: +- Removed 'num_objects' from trainModel(), this will get calculate based on how many class folders you have in your dataset directory/directories +- Removed 'transfer_with_full_learning' from trainModel() as it wasn't being used +- 'continue_from_model' now loads a full model and trains from that +- Changed 'enhance_data' to 'preprocess_layers' in trainModel() to be more fitting to the process happening +- Added 'show_training_graph' to trainModel() which is set to False by default. When set to True a graph plotting accuracy with validation accuracy as well as loss with validation loss will show at the end of training +- Moved preprocessing to before the selected model as this is what is recommended in official tensorflow and keras documentation and should improve accuracy when training +- Added RandomZoom() and RandomRotation() to preprocessing to further eliminate overfitting during training +- Removed ImageDataGenerator() as this is depreciated, proprocessing will take the place of this along with rescaling before each model +- Rescaling has been used instead of each models build in preprocessing function as there were issues with training and prediction when using them +- Removed flow_from_directory() as this has been depreceated and image_dataset_from_directory will take its place (this also has the functionality of automatically splitting a single dataset into training and validation datasets automatically) +- Added caching and prefetching of datasets to speed up the training process +- Setting up the chosen model has been simplified and updated to match current best practices in TensorFlow +- On all models 'include_top' has been set to false because of errors when it was set to True +- Saved weights now also contain the name of the model used during training as well as validation accuracy +- Removed tensorboard callback as it wasn't being used + +Custom Image Classification: +- set_json_path() is no longer required as long as the json file for the corresponding model is located in the 'json' folder and is named 'model_class.json' +- Added get_json() which will check if json path is set, if not set will check for 'json/model_class.json' and raise and error if neither are found + +Loading Model: +- Simplified code when setting up the model for prediction + +Classify Image: +- Changed all keras.preprocessing.image to keras.utils as keras.preprocessing is deprecated +- Added extra processing of the prediction with tf.nn.softmax() as raw data was unreadable +- Unfortunantly due to updates in Tensorflow models from previous generations of ImageAI are no longer going to be supported as the specific model used during training needs to be used during prediction. If you have a Tensorflow SavedModel or HDF5 full model, it can now be used for prediction regardless of if the model was trained in ImageAI or not. If you do not have the 'save_full_model' set to true on models you'd like to continue using, run a few epochs of transfer learning with that model and the 'save_full_model' set to True and it can be carried over to ImageAI V2.2.0 +(This is still being looked into to see if there is a way to transfer models over) \ No newline at end of file diff --git a/imageai/Classification/CUSTOMCLASSIFICATION.md b/imageai/Classification/Custom/Custom Classification.md similarity index 51% rename from imageai/Classification/CUSTOMCLASSIFICATION.md rename to imageai/Classification/Custom/Custom Classification.md index bffcc4ef..df3a7d70 100644 --- a/imageai/Classification/CUSTOMCLASSIFICATION.md +++ b/imageai/Classification/Custom/Custom Classification.md @@ -9,26 +9,26 @@ that you have trained the model on. ### TABLE OF CONTENTS -- :white_square_button: Custom Model Prediction -- :white_square_button: Custom Model Prediction with Full Model (NEW) -- :white_square_button: Custom Prediction with multiple models (NEW) -- :white_square_button: Convert custom model to Tensorflow's format (NEW) -- :white_square_button: Convert custom model to DeepStack's format (NEW) - +- Changes in ImageAI V2.2.0 +- Custom Model Prediction +- Custom Model Prediction with Full Model (NEW) +- Custom Prediction with multiple models (NEW) +- Convert custom model to Tensorflow's format (NEW) +- Convert custom model to DeepStack's format (NEW) ### Custom Model Prediction
-In this example, we will be using the model trained for 20 experiments on **IdenProf**, a dataset of uniformed professionals and achieved 65.17% accuracy on the test dataset. +In this example, we will be using the model trained for 20 experiments on **IdenProf**, a dataset of uniformed professionals and achieved 72.0% accuracy on the test dataset. (You can use your own trained model and generated JSON file. This 'class' is provided mainly for the purpose to use your own custom models.) Download the ResNet model of the model and JSON files in links below: -- [**ResNet50**](https://github.com/OlafenwaMoses/ImageAI/releases/download/essentials-v5/idenprof_resnet_ex-056_acc-0.993062.h5) _(Size = 90.4 mb)_ -- [**IdenProf model_class.json file**](https://github.com/OlafenwaMoses/ImageAI/releases/download/essentials-v5/idenprof.json) +- [**ResNet50**](../../../examples/idenprof/model_resnet50_ex-017_acc-0.787_vacc0.747.h5) _(Size = 90.2 mb)_ +- [**IdenProf model_class.json file**](../../../examples/idenprof/idenprof.json) Great! Once you have downloaded this model file and the JSON file, start a new python project, and then copy the model file and the JSON file to your project folder where your python files (.py files) will be. -Download the image below, or take any image on your computer that include any of the following professionals(Chef, Doctor, Engineer, Farmer, Fireman, Judge, Mechanic, Pilot, Police and Waiter) and copy it to your python project's folder. +Download the image below, or take any image on your computer that include any of the following professionals(Chef, Doctor, Engineer, Farmer, Fireman, Judge, Mechanic, Pilot, Police, or Waiter) and copy it to your python project's folder. Then create a python file and give it a name; an example is **FirstCustomPrediction.py**. Then write the code below into the python file: @@ -41,12 +41,12 @@ import os execution_path = os.getcwd() prediction = CustomImageClassification() -prediction.setModelTypeAsResNet50() -prediction.setModelPath(os.path.join(execution_path, "idenprof_resnet_ex-056_acc-0.993062.h5")) -prediction.setJsonPath(os.path.join(execution_path, "idenprof.json")) -prediction.loadModel(num_objects=10) +prediction.set_model_type('ResNet50') +prediction.set_model_path(os.path.join(execution_path, "idenprof_resnet50_ex-019_acc-0.773_vacc0.743")) +prediction.set_json_path(os.path.join(execution_path, "idenprof.json")) +prediction.load_trained_model() -predictions, probabilities = prediction.classifyImage(os.path.join(execution_path, "4.jpg"), result_count=5) +predictions, probabilities = prediction.classify_image(os.path.join(execution_path, "4.jpg"), result_count=5) for eachPrediction, eachProbability in zip(predictions, probabilities): print(eachPrediction + " : " + eachProbability) @@ -54,15 +54,17 @@ for eachPrediction, eachProbability in zip(predictions, probabilities): **Sample Result:** -![Sample Result](../../data-images/4.jpg) +![Sample Result](../../../data-images/4.jpg) ``` -mechanic : 76.82620286941528 -chef : 10.106072574853897 -waiter : 4.036874696612358 -police : 2.6663416996598244 -pilot : 2.239348366856575 +mechanic : 18.120427429676056 +waiter : 9.952791035175323 +chef : 9.288407862186432 +pilot : 9.214989095926285 +doctor : 9.11235511302948 ``` +You will notice that the probabilities of each prediction are fairly low, this is due to the number of training cycles being only 20. We call these training cycles epochs and when training a new model from scratch 50 epochs is the minimum recommended with 100 epochs being a good place to start and 200 epochs being what you should strive for if you have the computing resources and time. + The code above works as follows: ```python from imageai.Classification.Custom import CustomImageClassification @@ -78,29 +80,26 @@ The above line obtains the path to the folder that contains your python file (in ```python prediction = CustomImageClassification() -prediction.setModelTypeAsResNet50() -prediction.setModelPath(os.path.join(execution_path, "idenprof_resnet_ex-056_acc-0.993062.h5")) -prediction.setJsonPath(os.path.join(execution_path, "idenprof.json")) -prediction.loadModel(num_objects=10) +preiction.set_model_type('ResNet50') +prediction.set_model_path(os.path.join(execution_path, "idenprof_resnet50_ex-019_acc-0.773_vacc0.743")) +prediction.set_json_path(os.path.join(execution_path, "idenprof.json")) +prediction.load_full_model() ``` -In the lines above, we created and instance of the `CustomImageClassification()` - class in the first line, then we set the model type of the prediction object to ResNet by caling the `.setModelTypeAsResNet50()` - in the second line, we set the model path of the prediction object to the path of the custom model file (`idenprof_resnet_ex-056_acc-0.993062.h5`) we copied to the python file folder - in the third line, we set the path to the model_class.json of the model, we load the model and parse the number of objected that can be predicted in the model. +In the lines above, we created and instance of the `CustomImageClassification()` class in the first line, then we set the model type of the prediction object to ResNet by caling the `.set_model_type('ResNet50')` (if `.set_model_type() was not called it would default to ResNet50`) in the second line, we set the model path of the prediction object to the path of the custom model file (`idenprof_resnet50_ex-019_acc-0.773_vacc0.743`) we copied to the python file folder in the third line, we set the path to the model_class.json of the model, and then we load the model. ```python -predictions, probabilities = prediction.classifyImage(os.path.join(execution_path, "4.jpg"), result_count=5) +predictions, probabilities = prediction.classify_image(os.path.join(execution_path, "4.jpg"), result_count=5) ``` -In the above line, we defined 2 variables to be equal to the function called to predict an image, which is the `.classifyImage()` function, into which we parsed the path to our image and also state the number of prediction results we want to have (values from 1 to 10 in this case) parsing `result_count=5`. The `.classifyImage()` function will return 2 array objects with the first (**predictions**) being an array of predictions and the second (**percentage_probabilities**) being an array of the corresponding percentage probability for each prediction. +In the above line, we defined 2 variables to be equal to the function called to predict an image, which is the `.classify_image()` function, into which we parsed the path to our image and also state the number of prediction results we want to have (values from 1 to 10 in this case) parsing `result_count=5`. The `.classify_image()` function will return 2 array objects with the first (**predictions**) being an array of predictions and the second (**probabilities**) being an array of the corresponding percentage probability for each prediction. ```python for eachPrediction, eachProbability in zip(predictions, probabilities): print(eachPrediction + " : " + eachProbability) ``` -The above line obtains each object in the **predictions** array, and also obtains the corresponding percentage probability from the **percentage_probabilities**, and finally prints the result of both to console. +The above line obtains each object in the **predictions** array, and also obtains the corresponding percentage probability from the **probabilities**, and finally prints the result of both to console. **CustomImageClassification** class also supports the multiple predictions, input types and prediction speeds that are contained in the **ImageClassification** class. Follow this [link](README.md) to see all the details. @@ -122,34 +121,34 @@ import os execution_path = os.getcwd() predictor = CustomImageClassification() -predictor.setModelPath(model_path=os.path.join(execution_path, "idenprof_resnet.h5")) -predictor.setJsonPath(model_json=os.path.join(execution_path, "idenprof.json")) -predictor.setModelTypeAsResNet50() -predictor.loadModel(num_objects=10) +predictor.set_model_path(model_path=os.path.join(execution_path, "idenprof_resnet.h5")) +predictor.set_json_path(model_json=os.path.join(execution_path, "idenprof.json")) +predictor.set_model_type('ResNet50') +predictor.load_model() predictor2 = CustomImageClassification() -predictor2.setModelPath(model_path=os.path.join(execution_path, "idenprof_inception_0.719500.h5")) -predictor2.setJsonPath(model_json=os.path.join(execution_path, "idenprof.json")) +predictor2.set_model_path(model_path=os.path.join(execution_path, "idenprof_inception_0.719500.h5")) +predictor2.set_json_path(model_json=os.path.join(execution_path, "idenprof.json")) predictor2.setModelTypeAsInceptionV3() -predictor2.loadModel(num_objects=10) +predictor2.load_model(num_objects=10) -results, probabilities = predictor.classifyImage(image_input=os.path.join(execution_path, "9.jpg"), result_count=5) +results, probabilities = predictor.classify_image(image_input=os.path.join(execution_path, "9.jpg"), result_count=5) print(results) print(probabilities) -results2, probabilities2 = predictor3.classifyImage(image_input=os.path.join(execution_path, "9.jpg"), +results2, probabilities2 = predictor2.classify_image(image_input=os.path.join(execution_path, "9.jpg"), result_count=5) print(results2) print(probabilities2) print("-------------------------------") ``` -### Documentation + diff --git a/imageai/Classification/CUSTOMTRAINING.md b/imageai/Classification/Custom/Custom Training.md similarity index 62% rename from imageai/Classification/CUSTOMTRAINING.md rename to imageai/Classification/Custom/Custom Training.md index 1314b0ec..039ba954 100644 --- a/imageai/Classification/CUSTOMTRAINING.md +++ b/imageai/Classification/Custom/Custom Training.md @@ -3,25 +3,23 @@ --- **ImageAI** provides the most simple and powerful approach to training custom image prediction models -using state-of-the-art SqueezeNet, ResNet50, InceptionV3 and DenseNet -which you can load into the `imageai.Classification.Custom.CustomImageClassification` class. This allows - you to train your own model on any set of images that corresponds to any type of objects/persons. +using state-of-the-art MobileNetV2, ResNet50, InceptionV3 and DenseNet121 which you can load into the `imageai.Classification.Custom.CustomImageClassification` class. This allows +you to train your own model on any set of images that corresponds to any type of objects/persons. The training process generates a JSON file that maps the objects types in your image dataset and creates lots of models. You will then pick the model with the highest accuracy and perform custom image prediction using the model and the JSON file generated. ### TABLE OF CONTENTS -- :white_square_button: Custom Model Training Prediction -- :white_square_button: Saving Full Custom Model -- :white_square_button: Training on the IdenProf Dataset -- :white_square_button: Continuous Model Training -- :white_square_button: Transfer Learning (Training from a pre-trained model) - +- :white_square_button: Custom Model Training Prediction +- :white_square_button: Saving Full Custom Model +- :white_square_button: Training on the IdenProf Dataset +- :white_square_button: Continuous Model Training +- :white_square_button: Transfer Learning (Training from a pre-trained model) ### Custom Model Training -
+
-Because model training is a compute intensive tasks, we strongly advise you perform this experiment using a computer with a NVIDIA GPU and the GPU version of Tensorflow installed. Performing model training on CPU will my take hours or days. With NVIDIA GPU powered computer system, this will take a few hours. You can use Google Colab for this experiment as it has an NVIDIA K80 GPU available. +Because model training is a compute intensive tasks, we strongly advise you perform this experiment using a computer with a NVIDIA GPU and the GPU version of Tensorflow installed. Performing model training on CPU may take hours or days. With NVIDIA GPU powered computer system, this will take a few hours, maybe less. You can use Google Colab for this experiment as it has an NVIDIA K80 GPU available. To train a custom prediction model, you need to prepare the images you want to use to train the model. You will prepare the images as follows: @@ -29,11 +27,11 @@ You will prepare the images as follows: 1. Create a dataset folder with the name you will like your dataset to be called (e.g pets) 2. In the dataset folder, create a folder by the name **train** 3. In the dataset folder, create a folder by the name **test** -4. In the train folder, create a folder for each object you want to the model to predict and give the folder a name that corresponds to the respective object name (e.g dog, cat, squirrel, snake) +4. In the train folder, create a folder for each object you want the model to predict and give the folder a name that corresponds to the respective object name (e.g dog, cat, squirrel, snake) 5. In the test folder, create a folder for each object you want to the model to predict and give the folder a name that corresponds to the respective object name (e.g dog, cat, squirrel, snake) -6. In each folder present in the train folder, put the images of each object in its respective folder. This images are the ones to be used to train the model To produce a model that can perform well in practical applications, I recommend you about 500 or more images per object. 1000 images per object is just great -7. In each folder present in the test folder, put about 100 to 200 images of each object in its respective folder. These images are the ones to be used to test the model as it trains +6. In each folder present in the train folder, put the images of each object in its respective folder. These images are what the model will train off of. To produce a model that can perform well in practical applications, I recommend you use at least 500 or more images per object. 1000 images per object is ideal +7. In each folder present in the test folder, put about 100 to 200 images of each object in its respective folder. These images are the ones to be used to test the model as it trains. Ideally your test dataset will be around 20% of your entire dataset 8. Once you have done this, the structure of your image dataset folder should look like below: ``` pets//train//dog//dog-train-images @@ -49,38 +47,35 @@ You will prepare the images as follows: ```python from imageai.Classification.Custom import ClassificationModelTrainer model_trainer = ClassificationModelTrainer() - model_trainer.setModelTypeAsResNet50() - model_trainer.setDataDirectory("pets") - model_trainer.trainModel(num_objects=4, num_experiments=100, enhance_data=True, batch_size=32, show_network_summary=True) + model_trainer.set_model_type('ResNet50') + model_trainer.set_data_directory('pets') + model_trainer.train_model(num_experiments=100, preprocess_layers=True, show_network_summary=True) ``` - Yes! Just 5 lines of code and you can train any of the available 4 state-of-the-art Deep Learning algorithms on your custom dataset. +Yes! Just 4 lines of code and you can train any of the available 4 state-of-the-art Deep Learning algorithms on your custom dataset. Now lets take a look at how the code above works. ```python from imageai.Classification.Custom import ClassificationModelTrainer model_trainer = ClassificationModelTrainer() -model_trainer.setModelTypeAsResNet50() -model_trainer.setDataDirectory("pets") +model_trainer.set_model_type('ResNet50') +model_trainer.set_data_directory('pets') ``` In the first line, we import the **ImageAI** model training class, then we define the model trainer in the second line, - we set the network type in the third line and set the path to the image dataset we want to train the network on. +we set the network type in the third line and set the path to the image dataset we want to train the network on. ```python -model_trainer.trainModel(num_objects=4, num_experiments=100, enhance_data=True, batch_size=32, show_network_summary=True) +model_trainer.train_model(num_experiments=100, preprocess_layers=True, show_network_summary=True) ``` In the code above, we start the training process. The parameters stated in the function are as below: -- **num_objects** : this is to state the number of object types in the image dataset - **num_experiments** : this is to state the number of times the network will train over all the training images, - which is also called epochs -- **enhance_data (optional)** : This is used to state if we want the network to produce modified copies of the training -images for better performance. -- **batch_size** : This is to state the number of images the network will process at ones. The images - are processed in batches until they are exhausted per each experiment performed. + which is also called epochs +- **preprocess_layers (optional)** : This is used to state if we want the network to produce modified copies of the training + images for better performance. - **show_network_summary** : This is to state if the network should show the structure of the training - network in the console. + network in the console. When you start the training, you should see something like this in the console: @@ -129,17 +124,17 @@ Epoch 1/100 Let us explain the details shown above: 1. The line **Epoch 1/100** means the network is training the first experiment of the targeted 100 -2. The line `1/25 [>.............................] - ETA: 52s - loss: 2.3026 - acc: 0.2500` represents the number of batches that has been trained in the present experiment -3. The line `Epoch 00000: saving model to C:\Users\User\PycharmProjects\ImageAITest\pets\models\model_ex-000_acc-0.100000.h5` refers to the model saved after the present experiment. The **ex_000** represents the experiment at this stage while the **acc_0.100000** and **val_acc: 0.1000** represents the accuracy of the model on the test images after the present experiment (maximum value value of accuracy is 1.0). This result helps to know the best performed model you can use for custom image prediction. +2. The line `1/25 [>.............................] - ETA: 52s - loss: 2.3026 - acc: 0.2500` represents the number of batches that has been trained in the present experiment as well as the loss and how accurately the model is guessing the images +3. The line `Epoch 00000: saving model to C:\Users\User\PycharmProjects\ImageAITest\pets\models\model_ex-000_acc-0.100_vacc_0.100.h5` refers to the model saved after the present experiment. The **ex_000** represents the experiment at this stage while the **acc_0.1000** and **val_acc: 0.100** represents the accuracy of the model on the test images after the present experiment (maximum value of accuracy is 1.0). This result helps to know the best performed model you can use for custom image prediction. - Once you are done training your custom model, you can use the "CustomImagePrediction" class to perform image prediction with your model. Simply follow the link below. -[imageai/Classification/CUSTOMCLASSIFICATION.md](https://github.com/OlafenwaMoses/ImageAI/blob/master/imageai/Classification/CUSTOMCLASSIFICATION.md) + Once you are done training your custom model, you can use the 'CustomImagePrediction' class to perform image prediction with your model. Simply follow the link below. +[imageai/Classification/CUSTOMCLASSIFICATION.md](Custom Classification.md) ### Training on the IdenProf data A sample from the IdenProf Dataset used to train a Model for predicting professionals. -![](../../data-images/idenprof.jpg) +![](../../../data-images/idenprof.jpg) Below we provide a sample code to train on **IdenProf**, a dataset which contains images of 10 uniformed professionals. The code below will download the dataset and initiate the training: @@ -153,13 +148,13 @@ from imageai.Classification.Custom import ClassificationModelTrainer execution_path = os.getcwd() -TRAIN_ZIP_ONE = os.path.join(execution_path, "idenprof-train1.zip") -TRAIN_ZIP_TWO = os.path.join(execution_path, "idenprof-train2.zip") -TEST_ZIP = os.path.join(execution_path, "idenprof-test.zip") +TRAIN_ZIP_ONE = os.path.join(execution_path, 'idenprof-train1.zip') +TRAIN_ZIP_TWO = os.path.join(execution_path, 'idenprof-train2.zip') +TEST_ZIP = os.path.join(execution_path, 'idenprof-test.zip') -DATASET_DIR = os.path.join(execution_path, "idenprof") -DATASET_TRAIN_DIR = os.path.join(DATASET_DIR, "train") -DATASET_TEST_DIR = os.path.join(DATASET_DIR, "test") +DATASET_DIR = os.path.join(execution_path, 'idenprof') +DATASET_TRAIN_DIR = os.path.join(DATASET_DIR, 'train') +DATASET_TEST_DIR = os.path.join(DATASET_DIR, 'test') if(os.path.exists(DATASET_DIR) == False): os.mkdir(DATASET_DIR) @@ -170,52 +165,52 @@ if(os.path.exists(DATASET_TEST_DIR) == False): if(len(os.listdir(DATASET_TRAIN_DIR)) < 10): if(os.path.exists(TRAIN_ZIP_ONE) == False): - print("Downloading idenprof-train1.zip") - data = requests.get("https://github.com/OlafenwaMoses/IdenProf/releases/download/v1.0/idenprof-train1.zip", stream = True) - with open(TRAIN_ZIP_ONE, "wb") as file: + print('Downloading idenprof-train1.zip') + data = requests.get('https://github.com/OlafenwaMoses/IdenProf/releases/download/v1.0/idenprof-train1.zip', stream = True) + with open(TRAIN_ZIP_ONE, 'wb') as file: shutil.copyfileobj(data.raw, file) del data if (os.path.exists(TRAIN_ZIP_TWO) == False): - print("Downloading idenprof-train2.zip") - data = requests.get("https://github.com/OlafenwaMoses/IdenProf/releases/download/v1.0/idenprof-train2.zip", stream=True) - with open(TRAIN_ZIP_TWO, "wb") as file: + print('Downloading idenprof-train2.zip') + data = requests.get('https://github.com/OlafenwaMoses/IdenProf/releases/download/v1.0/idenprof-train2.zip', stream=True) + with open(TRAIN_ZIP_TWO, 'wb') as file: shutil.copyfileobj(data.raw, file) del data - print("Extracting idenprof-train1.zip") + print('Extracting idenprof-train1.zip') extract1 = ZipFile(TRAIN_ZIP_ONE) extract1.extractall(DATASET_TRAIN_DIR) extract1.close() - print("Extracting idenprof-train2.zip") + print('Extracting idenprof-train2.zip') extract2 = ZipFile(TRAIN_ZIP_TWO) extract2.extractall(DATASET_TRAIN_DIR) extract2.close() if(len(os.listdir(DATASET_TEST_DIR)) < 10): if (os.path.exists(TEST_ZIP) == False): - print("Downloading idenprof-test.zip") - data = requests.get("https://github.com/OlafenwaMoses/IdenProf/releases/download/v1.0/idenprof-test.zip", stream=True) - with open(TEST_ZIP, "wb") as file: + print('Downloading idenprof-test.zip') + data = requests.get('https://github.com/OlafenwaMoses/IdenProf/releases/download/v1.0/idenprof-test.zip', stream=True) + with open(TEST_ZIP, 'wb') as file: shutil.copyfileobj(data.raw, file) del data - print("Extracting idenprof-test.zip") + print('Extracting idenprof-test.zip') extract = ZipFile(TEST_ZIP) extract.extractall(DATASET_TEST_DIR) extract.close() model_trainer = ClassificationModelTrainer() -model_trainer.setModelTypeAsResNet50() -model_trainer.setDataDirectory(DATASET_DIR) -model_trainer.trainModel(num_objects=10, num_experiments=100, enhance_data=True, batch_size=32, show_network_summary=True) +model_trainer.set_model_type('ResNet50') +model_trainer.set_data_directory(DATASET_DIR) +model_trainer.train_model(num_experiments=100, preprocess_layers=True, show_network_summary=True) ``` ### Continuous Model Training -
+
**ImageAI** now allows you to continue training your custom model on your previously saved model. This is useful in cases of incomplete training due compute time limits/large size of dataset or should you intend to further train your model. Kindly note that **continuous training** is for using a previously saved model to train on the same dataset the model was trained on. -All you need to do is specify the `continue_from_model` parameter to the path of the previously saved model in your `trainModel()` function. +All you need to do is specify the `continue_from_model` parameter to the path of the previously saved model in your `train_model()` function. The model saved must be a full model, not just weights. See an example code below. ```python @@ -223,43 +218,43 @@ from imageai.Classification.Custom import ClassificationModelTrainer import os trainer = ClassificationModelTrainer() -trainer.setModelTypeAsDenseNet121() -trainer.setDataDirectory("idenprof") -trainer.trainModel(num_objects=10, num_experiments=50, enhance_data=True, batch_size=8, show_network_summary=True, continue_from_model="idenprof_densenet-0.763500.h5") +trainer.set_model_type('DenseNet121') +trainer.set_data_directory('idenprof') +trainer.train_model(num_experiments=50, preprocess_layers=True, batch_size=8, show_network_summary=True, continue_from_model='idenprof_densenet-0.763500.h5') ``` -### Transfer Learning (Training from a pre-trained model) -
+ -### Contact Developer + -### Documentation + diff --git a/imageai/Classification/Custom/__init__.py b/imageai/Classification/Custom/__init__.py index 4ba8222e..45f9fb23 100644 --- a/imageai/Classification/Custom/__init__.py +++ b/imageai/Classification/Custom/__init__.py @@ -1,87 +1,81 @@ +#!/usr/bin/python3 + +''' +Init file for custom classification in the ImageAI package +''' + +import os +import warnings +import json +import time + import tensorflow as tf from PIL import Image -import time import numpy as np -import os -import warnings from matplotlib.cbook import deprecated -import json +import matplotlib.pyplot as plt + +# pylint: disable=too-many-instance-attributes class ClassificationModelTrainer: - """ - This is the Classification Model training class, that allows you to define a deep learning network - from the 4 available networks types supported by ImageAI which are MobileNetv2, ResNet50, - InceptionV3 and DenseNet121. - """ + + ''' + This is the Classification Model training class, that allows you to + define a deep learning network from the 4 available networks types + supported by ImageAI which are MobileNetv2, ResNet50, InceptionV3 and + DenseNet121. + ''' def __init__(self): - self.__modelType = "" - self.__use_pretrained_model = False - self.__data_dir = "" - self.__train_dir = "" - self.__test_dir = "" - self.__logs_dir = "" + self.__model_type = '' + self.__data_dir = '' + self.__train_dir = '' + self.__test_dir = '' + self.__logs_dir = '' self.__num_epochs = 10 - self.__trained_model_dir = "" - self.__model_class_dir = "" + self.__trained_model_dir = '' + self.__model_class_dir = '' self.__initial_learning_rate = 1e-3 - self.__model_collection = [] - - - def setModelTypeAsSqueezeNet(self): - raise ValueError("ImageAI no longer support SqueezeNet. You can use MobileNetV2 instead by downloading the MobileNetV2 model and call the function 'setModelTypeAsMobileNetV2'") - def setModelTypeAsMobileNetV2(self): - """ - 'setModelTypeAsMobileNetV2()' is used to set the model type to the MobileNetV2 model - for the training instance object . - :return: - """ - self.__modelType = "mobilenetv2" - - @deprecated(since="2.1.6", message="'.setModelTypeAsResNet()' has been deprecated! Please use 'setModelTypeAsResNet50()' instead.") - def setModelTypeAsResNet(self): - return self.setModelTypeAsResNet50() - - def setModelTypeAsResNet50(self): - """ - 'setModelTypeAsResNet()' is used to set the model type to the ResNet model - for the training instance object . - :return: - """ - self.__modelType = "resnet50" - - - @deprecated(since="2.1.6", message="'.setModelTypeAsDenseNet()' has been deprecated! Please use 'setModelTypeAsDenseNet121()' instead.") - def setModelTypeAsDenseNet(self): - return self.setModelTypeAsDenseNet121() - - def setModelTypeAsDenseNet121(self): - """ - 'setModelTypeAsDenseNet()' is used to set the model type to the DenseNet model - for the training instance object . - :return: - """ - self.__modelType = "densenet121" - - def setModelTypeAsInceptionV3(self): - """ - 'setModelTypeAsInceptionV3()' is used to set the model type to the InceptionV3 model - for the training instance object . - :return: - """ - self.__modelType = "inceptionv3" - - def setDataDirectory(self, data_directory="", train_subdirectory="train", test_subdirectory="test", - models_subdirectory="models", json_subdirectory="json"): - """ - 'setDataDirectory()' - - - data_directory , is required to set the path to which the data/dataset to be used for - training is kept. The directory can have any name, but it must have 'train' and 'test' - sub-directory. In the 'train' and 'test' sub-directories, there must be sub-directories - with each having it's name corresponds to the name/label of the object whose images are - to be kept. The structure of the 'test' and 'train' folder must be as follows: + def set_model_type(self, model_type): + ''' + This is a function that simply sets the model type. It was designed + with future compatibility, making it easy to switch out, update, + and remove models as they're updated and changed + + Take the model type input and make it lowercase to remove any + capitalizaton errors then set model type variable + ''' + match model_type.lower(): + case 'mobilenetv2': + self.__model_type = 'MobileNetV2' + case 'resnet50': + self.__model_type = 'ResNet50' + case 'densenet121': + self.__model_type = 'DenseNet121' + case 'inceptionv3': + self.__model_type = 'InceptionV3' + # pylint: disable=too-many-arguments + + def set_data_directory( + self, + data_directory='', + train_subdirectory='train', + test_subdirectory='test', + models_subdirectory='models', + json_subdirectory='json', + logs_subdirectory='logs'): + ''' + 'set_data_directory()' + + - data_directory (optional), will set the path to which the + data/dataset to be used for training is kept. The directory + can have any name, but it must have 'train' and 'test' + sub-directory. In the 'train' and 'test' sub-directories, + there must be sub-directories with each having it's name + corresponds to the name/label of the object whose images are + to be kept. The structure of the 'test' and 'train' folder must + be as follows: >> train >> class1 >> class1_train_images >> class2 >> class2_train_images @@ -95,10 +89,36 @@ def setDataDirectory(self, data_directory="", train_subdirectory="train", test_s >> class4 >> class4_test_images >> class5 >> class5_test_images - - train_subdirectory (optional), subdirectory within 'data_directory' where the training set is. Defaults to 'train'. - - test_subdirectory (optional), subdirectory within 'data_directory' where the testing set is. Defaults to 'test'. - - models_subdirectory (optional), subdirectory within 'data_directory' where the output models will be saved. Defaults to 'models'. - - json_subdirectory (optional), subdirectory within 'data_directory' where the model classes json file will be saved. Defaults to 'json'. + Defaults to root project folder. + + - single_dataset_directory (optional), when set to True only one + folder will be looked for for a dataset and when set to False will + look for two directories (see data_directory). When this is set to + true the structure of the dataset folder must be as follows: + + >> data >> class1 >> class1_train_images + >> class2 >> class2_train_images + >> class3 >> class3_train_images + >> class4 >> class4_train_images + >> class5 >> class5_train_images + + Defaults to False. + + - dataset_directory (optional), subdirectory within 'data_directory' + where the data set is, is only used if 'single_dataset_directory' is + set to True. Defaults to 'data'. + - validation_split (optional), this number must be between 0 and 1 and + will deicde how the dataset is split up if 'single_dataset_directory' + is set to True. Ex: when set to 0.2 20% of the dataset will be used + for testing and 80% will be used for training. Defaults to 0.2. + - train_subdirectory (optional), subdirectory within 'data_directory' + where the training set is. Defaults to 'train'. + - test_subdirectory (optional), subdirectory within 'data_directory' + where the testing set is. Defaults to 'test'. + - models_subdirectory (optional), subdirectory within 'data_directory' + where the output models will be saved. Defaults to 'models'. + - json_subdirectory (optional), subdirectory within 'data_directory' + where the model classes json file will be saved. Defaults to 'json'. :param data_directory: :param train_subdirectory: @@ -106,22 +126,26 @@ def setDataDirectory(self, data_directory="", train_subdirectory="train", test_s :param models_subdirectory: :param json_subdirectory: :return: - """ + ''' self.__data_dir = data_directory self.__train_dir = os.path.join(self.__data_dir, train_subdirectory) self.__test_dir = os.path.join(self.__data_dir, test_subdirectory) - self.__trained_model_dir = os.path.join(self.__data_dir, models_subdirectory) - self.__model_class_dir = os.path.join(self.__data_dir, json_subdirectory) - self.__logs_dir = os.path.join(self.__data_dir, "logs") + self.__trained_model_dir = os.path.join( + self.__data_dir, models_subdirectory) + self.__model_class_dir = os.path.join( + self.__data_dir, json_subdirectory) + self.__logs_dir = os.path.join(self.__data_dir, logs_subdirectory) def lr_schedule(self, epoch): + ''' + This function sets the learning rate throughout model training in an + attempt to increase accuracy + ''' # Learning Rate Schedule - - - lr = self.__initial_learning_rate + learning_rate = self.__initial_learning_rate total_epochs = self.__num_epochs check_1 = int(total_epochs * 0.9) @@ -130,537 +154,756 @@ def lr_schedule(self, epoch): check_4 = int(total_epochs * 0.4) if epoch > check_1: - lr *= 1e-4 + learning_rate *= 1e-4 elif epoch > check_2: - lr *= 1e-3 + learning_rate *= 1e-3 elif epoch > check_3: - lr *= 1e-2 + learning_rate *= 1e-2 elif epoch > check_4: - lr *= 1e-1 - - - return lr - + learning_rate *= 1e-1 + + return learning_rate + + def build_model( + self, + input_shape, + model_type, + preprocess_layers, + num_classes): + ''' + Function to build a new model for training + ''' + + # Preprocess layers for image scaling + # If enhance data is set to True add extra preprocessing for better + # accuracy + + def preprocess_inputs(inputs): + if preprocess_layers: + processed = tf.keras.layers.RandomFlip( + 'horizontal', input_shape=input_shape)(inputs) + processed = tf.keras.layers.RandomRotation(0.1)(processed) + processed = tf.keras.layers.RandomZoom(0.1)(processed) + processed = tf.keras.layers.RandomHeight(0.1)(processed) + processed = tf.keras.layers.RandomWidth(0.1)(processed) + + return processed + + # Set up inputs and preprocess them + inputs = tf.keras.Input(shape=input_shape) + outputs = inputs + if preprocess_layers: + print('Using Enhanced Data Generation') + outputs = preprocess_inputs(inputs) + + if model_type == '': + print('No model type specified, defaulting to ResNet50') + model_type = 'ResNet50' + + # Check for model type and build a model based on it + match model_type: + case 'MobileNetV2': + outputs = tf.keras.layers\ + .Rescaling(1. / 127.5, + input_shape=input_shape, + offset=-1)(outputs) + model_name = 'model_mobilenetv2_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}.h5' + outputs = tf.keras.applications\ + .mobilenet_v2.MobileNetV2(input_shape=input_shape, + weights=None, + classes=num_classes, + include_top=False, + pooling='avg')(outputs) + case 'ResNet50': + outputs = tf.keras.layers\ + .Rescaling(1. / 255, + input_shape=input_shape)(outputs) + model_name = 'model_resnet50_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}.h5' + outputs = tf.keras.applications\ + .resnet50.ResNet50(input_shape=input_shape, + weights=None, + classes=num_classes, + include_top=False, + pooling='avg')(outputs) + case 'DenseNet121': + outputs = tf.keras.layers\ + .Rescaling(1. / 255, + input_shape=input_shape)(outputs) + model_name = 'model_densenet121_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}.h5' + outputs = tf.keras.applications\ + .densenet.DenseNet121(input_shape=input_shape, + weights=None, + classes=num_classes, + include_top=False, + pooling='avg')(outputs) + case 'InceptionV3': + outputs = tf.keras.layers\ + .Rescaling(1. / 127.5, + input_shape=input_shape, + offset=-1)(outputs) + model_name = 'model_inceptionv3_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}.h5' + outputs = tf.keras.applications\ + .inception_v3.InceptionV3(input_shape=input_shape, + weights=None, + classes=num_classes, + include_top=False, + pooling='avg')(outputs) + + # Build final output and create model + # outputs = tf.keras.layers.Dropout(0.2)(outputs) + outputs = tf.keras.layers.Dense( + num_classes, activation='softmax', use_bias=True)(outputs) + model = tf.keras.Model(inputs=inputs, outputs=outputs) + + # Compile the model to get ready for training + model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy( + ), optimizer='adam', metrics=['accuracy']) + + return model, model_name + + # pylint: disable=too-many-locals, too-many-branches, too-many-statements + def train_model( + self, + num_experiments=200, + preprocess_layers=False, + batch_size=32, + initial_learning_rate=1e-3, + show_network_summary=False, + training_image_size=224, + transfer_from_model=None, + continue_from_model=None, + save_weights_only=False, + show_training_graph=False): + ''' + 'train_model()' function starts the model actual training. It accepts + the following values: + + - num_experiments (optional), also known as epochs, it is the number + of times the network will train on all the training dataset + - preprocess_layers (optional) , this is used to modify the dataset + and create more instance of the training set to enhance the training + result + - batch_size (optional) , due to memory constraints, the network + trains on a batch at once, until all the training set is exhausted. + The value is set to 32 by default, but can be increased or decreased + depending on the meormory of the compute used for training. The + batch_size is conventionally set to 16, 32, 64, 128. + - initial_learning_rate (optional) , this value is used to adjust + the weights generated in the network. You rae advised to keep this + value as it is if you don't have deep understanding of this concept. + - show_network_summary (optional) , this value is used to show the + structure of the network should you desire to see it. It is set to + False by default + - training_image_size (optional) , this value is used to define the + image size on which the model will be trained. The value is 224 by + default and is kept at a minimum of 100. + - transfer_from_model (optional) , this is used to set the path to a + model file trained on another dataset. It is primarily used to + perform tramsfer learning. + - show_training_graph (optional), when set to True a graph plotting + accuracy with validation accuracy as well as loss with validation + loss at the end of training. It is set to False by default. + - keep_only_best (optional), when set to True all models saves (full + or just weights) other than the best will be deleted. Set to True by + default. - - def trainModel(self, num_objects, num_experiments=200, enhance_data=False, batch_size = 32, initial_learning_rate=1e-3, show_network_summary=False, training_image_size = 224, continue_from_model=None, transfer_from_model=None, transfer_with_full_training=True, initial_num_objects = None, save_full_model = False): - - """ - 'trainModel()' function starts the model actual training. It accepts the following values: - - num_objects , which is the number of classes present in the dataset that is to be used for training - - num_experiments , also known as epochs, it is the number of times the network will train on all the training dataset - - enhance_data (optional) , this is used to modify the dataset and create more instance of the training set to enhance the training result - - batch_size (optional) , due to memory constraints, the network trains on a batch at once, until all the training set is exhausted. The value is set to 32 by default, but can be increased or decreased depending on the meormory of the compute used for training. The batch_size is conventionally set to 16, 32, 64, 128. - - initial_learning_rate(optional) , this value is used to adjust the weights generated in the network. You rae advised to keep this value as it is if you don't have deep understanding of this concept. - - show_network_summary(optional) , this value is used to show the structure of the network should you desire to see it. It is set to False by default - - training_image_size(optional) , this value is used to define the image size on which the model will be trained. The value is 224 by default and is kept at a minimum of 100. - - continue_from_model (optional) , this is used to set the path to a model file trained on the same dataset. It is primarily for continuos training from a previously saved model. - - transfer_from_model (optional) , this is used to set the path to a model file trained on another dataset. It is primarily used to perform tramsfer learning. - - transfer_with_full_training (optional) , this is used to set the pre-trained model to be re-trained across all the layers or only at the top layers. - - initial_num_objects (required if 'transfer_from_model' is set ), this is used to set the number of objects the model used for transfer learning is trained on. If 'transfer_from_model' is set, this must be set as well. - - save_full_model ( optional ), this is used to save the trained models with their network types. Any model saved by this specification can be loaded without specifying the network type. - - - :param num_objects: :param num_experiments: - :param enhance_data: + :param preprocess_layers: :param batch_size: :param initial_learning_rate: :param show_network_summary: :param training_image_size: - :param continue_from_model: :param transfer_from_model: - :param initial_num_objects: :param save_full_model: :return: - """ + ''' + self.__num_epochs = num_experiments self.__initial_learning_rate = initial_learning_rate - lr_scheduler = tf.keras.callbacks.LearningRateScheduler(self.lr_schedule) + lr_scheduler = tf.keras.callbacks.LearningRateScheduler( + self.lr_schedule) - - if(training_image_size < 100): - warnings.warn("The specified training_image_size {} is less than 100. Hence the training_image_size will default to 100.".format(training_image_size)) + if training_image_size < 100: + warnings.warn(f'''The specified training_image_size + {training_image_size} is less than 100. Hence the + training_image_size will default to 100.''') training_image_size = 100 + input_shape = (training_image_size, training_image_size, 3) + # Set up training and validation datasets with caching and prefetching + # for improved training performance + autotune = tf.data.AUTOTUNE - if (self.__modelType == "mobilenetv2"): - if (continue_from_model != None): - model = tf.keras.applications.MobileNetV2(input_shape=(training_image_size, training_image_size, 3), weights=continue_from_model, classes=num_objects, - include_top=True) - if (show_network_summary == True): - print("Training using weights from a previouly model") - elif (transfer_from_model != None): - base_model = tf.keras.applications.MobileNetV2(input_shape=(training_image_size, training_image_size, 3), weights= transfer_from_model, - include_top=False, pooling="avg") - - network = base_model.output - network = tf.keras.layers.Dense(num_objects, activation='softmax', - use_bias=True)(network) - - model = tf.keras.model.Models(inputs=base_model.input, outputs=network) - - if (show_network_summary == True): - print("Training using weights from a pre-trained ImageNet model") - else: - base_model = tf.keras.applications.MobileNetV2(input_shape=(training_image_size, training_image_size, 3), weights= None, classes=num_objects, - include_top=False, pooling="avg") - - network = base_model.output - network = tf.keras.layers.Dense(num_objects, activation='softmax', - use_bias=True)(network) - - model = tf.keras.models.Model(inputs=base_model.input, outputs=network) - - elif (self.__modelType == "resnet50"): - if (continue_from_model != None): - model = tf.keras.applications.ResNet50(input_shape=(training_image_size, training_image_size, 3), weights=continue_from_model, classes=num_objects, - include_top=True) - if (show_network_summary == True): - print("Training using weights from a previouly model") - elif (transfer_from_model != None): - base_model = tf.keras.applications.ResNet50(input_shape=(training_image_size, training_image_size, 3), weights= transfer_from_model, - include_top=False, pooling="avg") - - network = base_model.output - network = tf.keras.layers.Dense(num_objects, activation='softmax', - use_bias=True)(network) - - model = tf.keras.model.Models(inputs=base_model.input, outputs=network) - - if (show_network_summary == True): - print("Training using weights from a pre-trained ImageNet model") - else: - base_model = tf.keras.applications.ResNet50(input_shape=(training_image_size, training_image_size, 3), weights= None, classes=num_objects, - include_top=False, pooling="avg") - - network = base_model.output - network = tf.keras.layers.Dense(num_objects, activation='softmax', - use_bias=True)(network) - - model = tf.keras.models.Model(inputs=base_model.input, outputs=network) - - elif (self.__modelType == "inceptionv3"): - - if (continue_from_model != None): - model = tf.keras.applications.InceptionV3(input_shape=(training_image_size, training_image_size, 3), weights=continue_from_model, classes=num_objects, - include_top=True) - if (show_network_summary == True): - print("Training using weights from a previouly model") - elif (transfer_from_model != None): - base_model = tf.keras.applications.InceptionV3(input_shape=(training_image_size, training_image_size, 3), weights= transfer_from_model, - include_top=False, pooling="avg") - - network = base_model.output - network = tf.keras.layers.Dense(num_objects, activation='softmax', - use_bias=True)(network) - - model = tf.keras.model.Models(inputs=base_model.input, outputs=network) - - if (show_network_summary == True): - print("Training using weights from a pre-trained ImageNet model") - else: - base_model = tf.keras.applications.InceptionV3(input_shape=(training_image_size, training_image_size, 3), weights= None, classes=num_objects, - include_top=False, pooling="avg") - - network = base_model.output - network = tf.keras.layers.Dense(num_objects, activation='softmax', - use_bias=True)(network) - - model = tf.keras.models.Model(inputs=base_model.input, outputs=network) - - base_model = tf.keras.applications.InceptionV3(input_shape=(training_image_size, training_image_size, 3), weights= None, classes=num_objects, - include_top=False, pooling="avg") - - elif (self.__modelType == "densenet121"): - if (continue_from_model != None): - model = tf.keras.applications.DenseNet121(input_shape=(training_image_size, training_image_size, 3), weights=continue_from_model, classes=num_objects, - include_top=True) - if (show_network_summary == True): - print("Training using weights from a previouly model") - elif (transfer_from_model != None): - base_model = tf.keras.applications.DenseNet121(input_shape=(training_image_size, training_image_size, 3), weights= transfer_from_model, - include_top=False, pooling="avg") - - network = base_model.output - network = tf.keras.layers.Dense(num_objects, activation='softmax', - use_bias=True)(network) - - model = tf.keras.model.Models(inputs=base_model.input, outputs=network) - - if (show_network_summary == True): - print("Training using weights from a pre-trained ImageNet model") - else: - base_model = tf.keras.applications.DenseNet121(input_shape=(training_image_size, training_image_size, 3), weights= None, classes=num_objects, - include_top=False, pooling="avg") - - network = base_model.output - network = tf.keras.layers.Dense(num_objects, activation='softmax', - use_bias=True)(network) - - model = tf.keras.models.Model(inputs=base_model.input, outputs=network) - - base_model = tf.keras.applications.DenseNet121(input_shape=(training_image_size, training_image_size, 3), weights= None, classes=num_objects, - include_top=False, pooling="avg") - - - optimizer = tf.keras.optimizers.Adam(lr=self.__initial_learning_rate, decay=1e-4) - model.compile(loss="categorical_crossentropy", optimizer=optimizer, metrics=["accuracy"]) - if (show_network_summary == True): - model.summary() + train_ds = tf.keras.utils.image_dataset_from_directory( + self.__train_dir, + seed=123, + image_size=(training_image_size, training_image_size), + batch_size=batch_size) - model_name = 'model_ex-{epoch:03d}_acc-{accuracy:03f}.h5' + val_ds = tf.keras.utils.image_dataset_from_directory( + self.__test_dir, + seed=123, + image_size=(training_image_size, training_image_size), + batch_size=batch_size) - log_name = '{}_lr-{}_{}'.format(self.__modelType, initial_learning_rate, time.strftime("%Y-%m-%d-%H-%M-%S")) + class_names = train_ds.class_names + num_classes = len(class_names) - if not os.path.isdir(self.__trained_model_dir): - os.makedirs(self.__trained_model_dir) + train_ds = train_ds.cache().shuffle(1000) + train_ds = train_ds.prefetch(buffer_size=autotune) + val_ds = val_ds.cache().prefetch(buffer_size=autotune) - if not os.path.isdir(self.__model_class_dir): - os.makedirs(self.__model_class_dir) + if continue_from_model is None: + model, model_name = self.build_model( + input_shape, self.__model_type, preprocess_layers, num_classes) - if not os.path.isdir(self.__logs_dir): - os.makedirs(self.__logs_dir) + # If continuing a model load those weights + if transfer_from_model is not None: + model.load_weights(transfer_from_model) + else: + model = tf.keras.models.load_model(continue_from_model) + model_names = ['mobilenet_v2', 'resnet50', + 'densenet121', 'inception_v3'] + model_name = '' + if self.__model_type == 'Custom': + model_name = 'model_custom_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}' + for layer in model.layers: + if layer.name in model_names: + if layer.name == model_names[0]: + model_name = 'model_mobilenetv2_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}.h5' + # for layers in model.layers[:100]: + # layer.trainable = False + elif layer.name == model_names[1]: + model_name = 'model_resnet50_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}.h5' + # for layers in model.layers[:100]: + # layer.trainable = False + elif layer.name == model_names[2]: + model_name = 'model_densenet121_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}.h5' + # for layers in model.layers[:100]: + # layer.trainable = False + elif layer.name == model_names[3]: + model_name = 'model_inceptionv3_ex-{epoch:03d}'\ + '_acc-{accuracy:.3f}_vacc{val_accuracy:.3f}.h5' + # for layers in model.layers[:100]: + # layer.trainable = False + break + + # Print model summary if set and print if using a previous model + if show_network_summary: + if transfer_from_model is not None: + print('Training using weights from a previous model') + if continue_from_model is not None: + print('Starting training from a previously trained model') + model.summary() + # Set templates for model name, model path, log name, and log path model_path = os.path.join(self.__trained_model_dir, model_name) - - - logs_path = os.path.join(self.__logs_dir, log_name) - if not os.path.isdir(logs_path): - os.makedirs(logs_path) - - save_weights_condition = True - - if(save_full_model == True ): - save_weights_condition = False - - - checkpoint = tf.keras.callbacks.ModelCheckpoint(filepath=model_path, - monitor='accuracy', - verbose=1, - save_weights_only=save_weights_condition, - save_best_only=True, - period=1) - - - tensorboard = tf.keras.callbacks.TensorBoard(log_dir=logs_path, - histogram_freq=0, - write_graph=False, - write_images=False) - - - if (enhance_data == True): - print("Using Enhanced Data Generation") - - height_shift = 0 - width_shift = 0 - if (enhance_data == True): - height_shift = 0.1 - width_shift = 0.1 - - train_datagen = tf.keras.preprocessing.image.ImageDataGenerator( - rescale=1. / 255, - horizontal_flip=enhance_data, height_shift_range=height_shift, width_shift_range=width_shift) - - test_datagen = tf.keras.preprocessing.image.ImageDataGenerator( - rescale=1. / 255) - - train_generator = train_datagen.flow_from_directory(self.__train_dir, target_size=(training_image_size, training_image_size), - batch_size=batch_size, - class_mode="categorical") - test_generator = test_datagen.flow_from_directory(self.__test_dir, target_size=(training_image_size, training_image_size), - batch_size=batch_size, - class_mode="categorical") - - class_indices = train_generator.class_indices + formatted_time = time.strftime('%Y-%m-%d-%H-%M-%S') + log_name = f'{self.__model_type}_lr-'\ + f'{initial_learning_rate}_{formatted_time}' + log_path = os.path.join(self.__logs_dir, log_name) + + # Check if data directories exist and if they don't create them + directories = [self.__trained_model_dir, self.__model_class_dir, + self.__logs_dir, log_path] + for dirs in directories: + if not os.path.isdir(dirs): + os.makedirs(dirs) + + # Create a checkpoint callback to save models as they're completed + checkpoint = tf.keras.callbacks\ + .ModelCheckpoint(filepath=model_path, + monitor='accuracy', + verbose=1, + save_weights_only=save_weights_only, + save_best_only=True) + + # Create a model class json file class_json = {} - for eachClass in class_indices: - class_json[str(class_indices[eachClass])] = eachClass - - with open(os.path.join(self.__model_class_dir, "model_class.json"), "w+") as json_file: - json.dump(class_json, json_file, indent=4, separators=(",", " : "), + for i, class_name in enumerate(class_names): + class_json[str(i)] = class_name + with open(os.path.join(self.__model_class_dir, 'model_class.json'), + 'w+', encoding='UTF-8') as json_file: + json.dump(class_json, json_file, indent=4, separators=(',', ' : '), ensure_ascii=True) - json_file.close() - print("JSON Mapping for the model classes saved to ", os.path.join(self.__model_class_dir, "model_class.json")) - - num_train = len(train_generator.filenames) - num_test = len(test_generator.filenames) - print("Number of experiments (Epochs) : ", self.__num_epochs) - - - model.fit_generator(train_generator, steps_per_epoch=int(num_train / batch_size), epochs=self.__num_epochs, - validation_data=test_generator, - validation_steps=int(num_test / batch_size), callbacks=[checkpoint, lr_scheduler]) + print('JSON Mapping for the model classes saved to ', + os.path.join(self.__model_class_dir, 'model_class.json')) + + # Print number of experiments that will be run + print('Number of experiments (Epochs) : ', self.__num_epochs) + + # Train the model + history = model.fit(train_ds, epochs=self.__num_epochs, + validation_data=val_ds, + callbacks=[checkpoint, lr_scheduler]) + # callbacks=[checkpoint]) + + if show_training_graph: + acc = history.history['accuracy'] + val_acc = history.history['val_accuracy'] + + loss = history.history['loss'] + val_loss = history.history['val_loss'] + + epochs_range = range(self.__num_epochs) + + plt.figure(figsize=(8, 8)) + plt.subplot(1, 2, 1) + plt.plot(epochs_range, acc, label='Training Accuracy') + plt.plot(epochs_range, val_acc, label='Validation Accuracy') + plt.legend(loc='lower right') + plt.title('Training and Validation Accuracy') + + plt.subplot(1, 2, 2) + plt.plot(epochs_range, loss, label='Training Loss') + plt.plot(epochs_range, val_loss, label='Validation Loss') + plt.legend(loc='upper right') + plt.title('Training and Validation Loss') + plt.show() + + def setModelTypeAsSqueezeNet(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + raise ValueError( + 'ImageAI no longer support SqueezeNet. You can use MobileNetV2' + ' instead by downloading the MobileNetV2 model and call the' + ' function \'setModelTypeAsMobileNetV2\'') + + @ deprecated(since='2.1.6', message='\'.setModelTypeAsResNet()\' has been' + ' deprecated! Please use' + ' \'set_model_type(\'ResNet50\')\' instead.') + def setModelTypeAsResNet(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.setModelTypeAsResNet50() + @ deprecated(since='2.1.6', message='\'.setModelTypeAsDenseNet()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'DenseNet121\')\' instead.') + def setModelTypeAsDenseNet(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('DenseNet121') + + @ deprecated(since='2.2.0', message='\'.setModelTypeAsMobileNetV2()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'MobileNetV2\')\' instead.') + def setModelTypeAsMobileNetV2(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('MobileNetV2') + + @ deprecated(since='2.2.0', message='\'.setModelTypeAsResNet50()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'ResNet50\')\' instead.') + def setModelTypeAsResNet50(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('ResNet50') + + @ deprecated(since='2.2.0', message='\'.setModelTypeAsDenseNet121()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'DenseNet121\')\' instead.') + def setModelTypeAsDenseNet121(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('DenseNet121') + + @ deprecated(since='2.2.0', message='\'.setModelTypeAsInceptionV3()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'InceptionV3\')\' instead.') + def setModelTypeAsInceptionV3(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('InceptionV3') +class CustomImageClassification: + ''' + This is the image classification class for custom models trained with the + 'ClassificationModelTrainer' class. It provides support for 4 different + models which are: ResNet50, MobileNetV2, DenseNet121 and Inception V3. + After instantiating this class, you can set it's properties and make image + classification using it's pre-defined functions. + + The following functions are required to be called before a classification + can be made + * set_model_path() , path to your custom model + * set_json_path , , path to your custom model's corresponding JSON file + * At least of of the following and it must correspond to the model set in + the set_model_path() [setModelTypeAsMobileNetV2(), + setModelTypeAsResNet50(), setModelTypeAsDenseNet121, + setModelTypeAsInceptionV3] + * load_trained_model() [This must be called once only before making a + classification] + + Once the above functions have been called, you can call the + classify_image() function of the classification instance object + at anytime to predict an image. + ''' -class CustomImageClassification: - """ - This is the image classification class for custom models trained with the 'ClassificationModelTrainer' class. It provides support for 4 different models which are: - ResNet50, MobileNetV2, DenseNet121 and Inception V3. After instantiating this class, you can set it's properties and - make image classification using it's pre-defined functions. - - The following functions are required to be called before a classification can be made - * setModelPath() , path to your custom model - * setJsonPath , , path to your custom model's corresponding JSON file - * At least of of the following and it must correspond to the model set in the setModelPath() - [setModelTypeAsMobileNetV2(), setModelTypeAsResNet50(), setModelTypeAsDenseNet121, setModelTypeAsInceptionV3] - * loadModel() [This must be called once only before making a classification] - - Once the above functions have been called, you can call the classifyImage() function of the classification instance - object at anytime to predict an image. - """ def __init__(self): - self.__modelType = "" - self.modelPath = "" - self.jsonPath = "" - self.numObjects = 10 - self.__model_classes = dict() - self.__modelLoaded = False + self.__model_type = '' + self.model_path = '' + self.json_path = '' + self.num_objects = 10 + self.__model_classes = {} + self.__model_loaded = False self.__model_collection = [] self.__input_image_size = 224 - - def setModelPath(self, model_path): - """ - 'setModelPath()' function is required and is used to set the file path to the model adopted from the list of the - available 4 model types. The model path must correspond to the model type set for the classification instance object. - :param model_path: - :return: - """ - self.modelPath = model_path - - def setJsonPath(self, model_json): - """ - 'setJsonPath()' + def set_model_path(self, model_path): + ''' + 'set_model_path()' function is required and is used to set the file + path to the model adopted from the list of the available 4 model + types. The model path must correspond to the model type set for the + classification instance object. :param model_path: :return: - """ - self.jsonPath = model_json + ''' - def setModelTypeAsMobileNetV2(self): - """ - 'setModelTypeAsMobileNetV2()' is used to set the model type to the MobileNetV2 model - for the classification instance object . - :return: - """ - self.__modelType = "mobilenetv2" + self.model_path = model_path - def setModelTypeAsResNet50(self): - """ - 'setModelTypeAsResNet50()' is used to set the model type to the ResNet50 model - for the classification instance object . - :return: - """ - self.__modelType = "resnet50" - - def setModelTypeAsDenseNet121(self): - """ - 'setModelTypeAsDenseNet121()' is used to set the model type to the DenseNet121 model - for the classification instance object . - :return: - """ - self.__modelType = "densenet121" + def set_json_path(self, model_json): + ''' + 'set_json_path()' function is not required unless model json is not + named 'model_class.json' in the 'json' directory of project root + folder. - def setModelTypeAsInceptionV3(self): - """ - 'setModelTypeAsInceptionV3()' is used to set the model type to the InceptionV3 model - for the classification instance object . + :param model_path: :return: - """ - self.__modelType = "inceptionv3" - - def loadModel(self, classification_speed="normal", num_objects=10): - """ - 'loadModel()' function is used to load the model structure into the program from the file path defined - in the setModelPath() function. This function receives an optional value which is "classification_speed". - The value is used to reduce the time it takes to classify an image, down to about 50% of the normal time, - with just slight changes or drop in classification accuracy, depending on the nature of the image. - * classification_speed (optional); Acceptable values are "normal", "fast", "faster" and "fastest" - - :param classification_speed : + ''' + + self.json_path = model_json + + def set_model_type(self, model_type): + ''' + Take the model type input and make it lowercase to remove any + capitalizaton errors then set model type variable + ''' + + match model_type.lower(): + case 'mobilenetv2': + self.__model_type = 'MobileNetV2' + case 'resnet50': + self.__model_type = 'ResNet50' + case 'densenet121': + self.__model_type = 'DenseNet121' + case 'inceptionv3': + self.__model_type = 'InceptionV3' + + def get_json(self): + ''' + If json path is specified open it, otherwise check the default + location, if json path is not specified nor in the default + location raise ValueError + ''' + + if self.json_path != '': + with open(self.json_path, 'r', encoding='UTF-8') as json_file: + self.__model_classes = json.load(json_file) + else: + try: + with open('json/model_class.json', 'r', encoding='UTF-8')\ + as json_file: + self.__model_classes = json.load(json_file) + except Exception as exc: + raise ValueError( + 'There was an error when loading the model class json\ + file. Make sure the file location is\ + \'json\\model_class.json\' or set the json path with\ + set_json_path()') from exc + + def load_trained_model(self, classification_speed='normal'): + ''' + 'load_trained_model()' function is used to load the model structure + into the program from the file path defined in the set_model_path() + function. This function receives an optional value which is + 'classification_speed'. The value is used to reduce the time it takes + to classify an image, down to about 50% of the normal time, with just + slight changes or drop in classification accuracy, depending on the + nature of the image. * classification_speed (optional); Acceptable + values are 'normal', 'fast', 'faster' and 'fastest' + + :param classification_speed: :return: - """ - - self.__model_classes = json.load(open(self.jsonPath)) - - if(classification_speed=="normal"): - self.__input_image_size = 224 - elif(classification_speed=="fast"): - self.__input_image_size = 160 - elif(classification_speed=="faster"): - self.__input_image_size = 120 - elif (classification_speed == "fastest"): - self.__input_image_size = 100 - - if (self.__modelLoaded == False): - - image_input = tf.keras.layers.Input(shape=(self.__input_image_size, self.__input_image_size, 3)) + ''' + + # Call get_json to load all model classes + self.get_json() + + # Adjust the size that the input image will be rescaled to + # Smaller numbers mean smaller picture which means quicker + # analysis and prediction + # The trade-off to faster speeds is prediction accuracy + match classification_speed: + case 'normal': + self.__input_image_size = 224 + case 'fast': + self.__input_image_size = 160 + case 'faster': + self.__input_image_size = 120 + case 'fastest': + self.__input_image_size = 100 + + input_shape = (self.__input_image_size, self.__input_image_size, 3) + + # Create model inputs then rescale inputs to image size which is + # used by all models + inputs = tf.keras.Input( + shape=(self.__input_image_size, self.__input_image_size, 3)) + rescaled = tf.keras.layers.Rescaling( + 1. / 255, input_shape=input_shape)(inputs) + + # Only load model if a model hasn't already been loaded + if not self.__model_loaded: + + # Get number of objects from model class list length + num_classes = len(self.__model_classes) + try: - if(self.__modelType == "" ): - raise ValueError("You must set a valid model type before loading the model.") + # Set outputs based on model type + if self.__model_type == '': + print('No model type specified, defaulting to ResNet50') + self.__model_type = 'ResNet50' + + match self.__model_type: + case 'MobileNetV2': + rescaled = tf.keras.layers\ + .Rescaling(1. / 127.5, + input_shape=input_shape, + offset=-1)(inputs) + outputs = tf.keras.applications\ + .mobilenet_v2.MobileNetV2(input_shape=input_shape, + weights=None, + classes=num_classes, + include_top=False, + pooling='avg')(rescaled) + case 'ResNet50': + rescaled = tf.keras.layers\ + .Rescaling(1. / 255, + input_shape=input_shape)(inputs) + outputs = tf.keras.applications\ + .resnet50.ResNet50(input_shape=input_shape, + weights=None, + classes=num_classes, + include_top=False, + pooling='avg')(rescaled) + case 'DenseNet121': + rescaled = tf.keras.layers\ + .Rescaling(1. / 255, + input_shape=input_shape)(inputs) + outputs = tf.keras.applications\ + .densenet.DenseNet121(input_shape=input_shape, + weights=None, + classes=num_classes, + include_top=True, + pooling='avg')(rescaled) + case 'InceptionV3': + rescaled = tf.keras.layers\ + .Rescaling(1. / 127.5, + input_shape=input_shape, + offset=-1)(inputs) + outputs = tf.keras.applications\ + .inception_v3.InceptionV3(input_shape=input_shape, + weights=None, + classes=num_classes, + include_top=False, + pooling='avg')(rescaled) + case _: + raise ValueError( + 'You must set a valid model type before' + ' loading the model.') + + # Create model, then load weights into the model + outputs = tf.keras.layers.Dense( + num_classes, activation='softmax', use_bias=True)(outputs) + model = tf.keras.Model(inputs=inputs, outputs=outputs) + model.load_weights(self.model_path) - elif(self.__modelType == "mobilenetv2"): - model = tf.keras.applications.MobileNetV2(input_shape=(self.__input_image_size, self.__input_image_size, 3), weights=self.modelPath, classes = num_objects ) self.__model_collection.append(model) - self.__modelLoaded = True - try: - None - except: - raise ValueError("An error occured. Ensure your model file is a MobileNetV2 Model and is located in the path {}".format(self.modelPath)) - - elif(self.__modelType == "resnet50"): - try: - model = tf.keras.applications.ResNet50(input_shape=(self.__input_image_size, self.__input_image_size, 3), weights=None, classes = num_objects ) - model.load_weights(self.modelPath) - self.__model_collection.append(model) - self.__modelLoaded = True - except: - raise ValueError("An error occured. Ensure your model file is a ResNet50 Model and is located in the path {}".format(self.modelPath)) - - elif (self.__modelType == "densenet121"): - try: - model = tf.keras.applications.DenseNet121(input_shape=(self.__input_image_size, self.__input_image_size, 3), weights=self.modelPath, classes = num_objects) - self.__model_collection.append(model) - self.__modelLoaded = True - except: - raise ValueError("An error occured. Ensure your model file is a DenseNet121 Model and is located in the path {}".format(self.modelPath)) - - elif (self.__modelType == "inceptionv3"): - try: - model = tf.keras.applications.InceptionV3(input_shape=(self.__input_image_size, self.__input_image_size, 3), weights=self.modelPath, classes = num_objects ) - self.__model_collection.append(model) - self.__modelLoaded = True - except: - raise ValueError("An error occured. Ensure your model file is in {}".format(self.modelPath)) - def loadFullModel(self, classification_speed="normal", num_objects=10): - """ - 'loadFullModel()' function is used to load the model structure into the program from the file path defined - in the setModelPath() function. As opposed to the 'loadModel()' function, you don't need to specify the model type. This means you can load any Keras model trained with or without ImageAI and perform image prediction. - - prediction_speed (optional), Acceptable values are "normal", "fast", "faster" and "fastest" - - num_objects (required), the number of objects the model is trained to recognize + self.__model_loaded = True + + except Exception as exc: + raise ValueError( + f'An error occured. Ensure your model file is a' + f' {self.__model_type} Model and is located in the' + f' path {self.model_path}') from exc + + def load_full_model(self, classification_speed='normal'): + ''' + 'load_full_model()' function is used to load the model structure into + the program from the file path defined in the set_model_path() + function. As opposed to the 'load_trained_model()' function, you don't + need to specify the model type. This means you can load any Keras + model trained with or without ImageAI and perform image prediction. + - prediction_speed (optional), Acceptable values are 'normal', 'fast', + 'faster' and 'fastest' :param prediction_speed: - :param num_objects: :return: - """ - - self.numObjects = num_objects - self.__model_classes = json.load(open(self.jsonPath)) - - if (classification_speed == "normal"): - self.__input_image_size = 224 - elif (classification_speed == "fast"): - self.__input_image_size = 160 - elif (classification_speed == "faster"): - self.__input_image_size = 120 - elif (classification_speed == "fastest"): - self.__input_image_size = 100 - - if (self.__modelLoaded == False): - - model = tf.keras.models.load_model(filepath=self.modelPath) - self.__model_collection.append(model) - self.__modelLoaded = True - self.__modelType = "full" + ''' - def getModels(self): - """ - 'getModels()' provides access to the internal model collection. Helpful if models are used down the line with tools like lime. - :return: - """ - return self.__model_collection + self.get_json() + # Adjust the size that the input image will be rescaled to smaller + # numbers mean smaller picture which means quicker analysis and + # prediction, the trade-off to faster speeds is prediction accuracy - def classifyImage(self, image_input, result_count=5, input_type="file"): - """ - 'classifyImage()' function is used to classify a given image by receiving the following arguments: - * input_type (optional) , the type of input to be parsed. Acceptable values are "file", "array" and "stream" - * image_input , file path/numpy array/image file stream of the image. - * result_count (optional) , the number of classifications to be sent which must be whole numbers between - 1 and 1000. The default is 5. + match classification_speed: + case 'normal': + self.__input_image_size = 224 + case 'fast': + self.__input_image_size = 160 + case 'faster': + self.__input_image_size = 120 + case 'fastest': + self.__input_image_size = 100 - This function returns 2 arrays namely 'classification_results' and 'classification_probabilities'. The 'classification_results' - contains possible objects classes arranged in descending of their percentage probabilities. The 'classification_probabilities' - contains the percentage probability of each object class. The position of each object class in the 'classification_results' - array corresponds with the positions of the percentage probability in the 'classification_probabilities' array. + # Only load model if a model hasn't already been loaded + if not self.__model_loaded: + + model = tf.keras.models.load_model(filepath=self.model_path) + self.__model_collection.append(model) + self.__model_loaded = True + self.__model_type = 'full' + + # pylint: disable=too-many-locals + def classify_image( + self, + image_input, + result_count=3, + input_type='file', + show_output=True): + ''' + 'classify_image()' function is used to classify a given image by + receiving the following arguments: + * input_type (optional) , the type of input to be parsed. Acceptable + values are 'file', 'array' and 'stream' + * image_input , file path/numpy array/image file stream of the image. + * result_count (optional) , the number of classifications to be sent + which must be whole numbers between 1 and 1000. The default is 5. + + This function returns 2 arrays namely 'classification_results' and + 'classification_probabilities'. The 'classification_results' contains + possible objects classes arranged in descending of their percentage + probabilities. The 'classification_probabilities' contains the + percentage probability of each object class. The position of each + object class in the 'classification_results' array corresponds with + the positions of the percentage probability in the + 'classification_probabilities' array. :param input_type: :param image_input: :param result_count: :return classification_results, classification_probabilities: - """ + ''' classification_results = [] classification_probabilities = [] - if (self.__modelLoaded == False): - raise ValueError("You must call the loadModel() function before making classification.") + if not self.__model_loaded: + raise ValueError( + 'You must call the load_trained_model() function before making' + ' classification.') + + if input_type == 'file': + image_to_predict = tf.keras.utils\ + .load_img(image_input, target_size=( + self.__input_image_size, self.__input_image_size)) + image_to_predict = tf.keras.utils.img_to_array( + image_to_predict) + image_to_predict = tf.expand_dims(image_to_predict, 0) + elif input_type == 'array': + image_to_predict = Image.fromarray(image_input) + image_to_predict = image_to_predict.resize( + (self.__input_image_size, self.__input_image_size)) + image_to_predict = tf.keras.utils.img_to_array( + image_to_predict) + image_to_predict = tf.expand_dims(image_to_predict, 0) + elif input_type == 'stream': + image_to_predict = Image.open(image_input) + image_to_predict = image_to_predict.resize( + (self.__input_image_size, self.__input_image_size)) + image_to_predict = tf.expand_dims(image_to_predict, axis=0) + image_to_predict = image_to_predict.copy() + image_to_predict = np.asarray( + image_to_predict, dtype=np.float64) + model = self.__model_collection[0] + prediction = model.predict( + image_to_predict, verbose=1 if show_output is True else 0) + + predictiondata = [] + for pred in prediction: + scores = tf.nn.softmax(pred) + top_indices = pred.argsort()[-result_count:][::-1] + for i in top_indices: + each_result = [] + each_result.append(self.__model_classes[str(i)]) + each_result.append(100 * scores[i].numpy()) + predictiondata.append(each_result) + + for result in predictiondata: + classification_results.append(str(result[0])) + classification_probabilities.append(str(result[1])) + + return classification_results, classification_probabilities + + def setModelTypeAsSqueezeNet(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + raise ValueError('''ImageAI no longer support SqueezeNet. You can use + MobileNetV2 instead by downloading the MobileNetV2 model and + call the function 'setModelTypeAsMobileNetV2\'''') + + @ deprecated(since='2.1.6', message='\'.predictImage()\' has been' + ' deprecated! Please use \'classify_image()\' instead.') + # pylint: disable=invalid-name + def predictImage(self, image_input, result_count=3, input_type='file'): + '''Function set in place to redirect users due to depreceation''' + return self.classify_image(image_input, result_count, input_type) + + @ deprecated(since='2.1.6', message='\'.setModelTypeAsResNet()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'ResNet50\')\' instead.') + def setModelTypeAsResNet(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.setModelTypeAsResNet50() - else: - if (input_type == "file"): - try: - image_to_predict = tf.keras.preprocessing.image.load_img(image_input, target_size=(self.__input_image_size, self.__input_image_size)) - image_to_predict = tf.keras.preprocessing.image.img_to_array(image_to_predict, data_format="channels_last") - image_to_predict = np.expand_dims(image_to_predict, axis=0) - except: - raise ValueError("You have set a path to an invalid image file.") - elif (input_type == "array"): - try: - image_input = Image.fromarray(np.uint8(image_input)) - image_input = image_input.resize((self.__input_image_size, self.__input_image_size)) - image_input = np.expand_dims(image_input, axis=0) - image_to_predict = image_input.copy() - image_to_predict = np.asarray(image_to_predict, dtype=np.float64) - except: - raise ValueError("You have parsed in a wrong numpy array for the image") - elif (input_type == "stream"): - try: - image_input = Image.open(image_input) - image_input = image_input.resize((self.__input_image_size, self.__input_image_size)) - image_input = np.expand_dims(image_input, axis=0) - image_to_predict = image_input.copy() - image_to_predict = np.asarray(image_to_predict, dtype=np.float64) - - except: - raise ValueError("You have parsed in a wrong stream for the image") - - if (self.__modelType == "mobilenetv2"): - image_to_predict = tf.keras.applications.mobilenet_v2.preprocess_input(image_to_predict) - elif (self.__modelType == "full"): - image_to_predict = tf.keras.applications.mobilenet_v2.preprocess_input(image_to_predict) - elif (self.__modelType == "inceptionv3"): - image_to_predict = tf.keras.applications.inception_v3.preprocess_input(image_to_predict) - elif (self.__modelType == "densenet121"): - image_to_predict = tf.keras.applications.densenet.preprocess_input(image_to_predict) - try: - model = self.__model_collection[0] - prediction = model.predict(image_to_predict, steps=1) - - predictiondata = [] - for pred in prediction: - top_indices = pred.argsort()[-result_count:][::-1] - for i in top_indices: - each_result = [] - each_result.append(self.__model_classes[str(i)]) - each_result.append(pred[i]) - predictiondata.append(each_result) - - for result in predictiondata: - classification_results.append(str(result[0])) - classification_probabilities.append(result[1] * 100) - - except: - raise ValueError("Error. Ensure your input image is valid") - - return classification_results, classification_probabilities - - - @deprecated(since="2.1.6", message="'.predictImage()' has been deprecated! Please use 'classifyImage()' instead.") - def predictImage(self, image_input, result_count=5, input_type="file"): - - return self.classifyImage(image_input, result_count, input_type) \ No newline at end of file + @ deprecated(since='2.1.6', message='\'.setModelTypeAsDenseNet()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'DenseNet121\')\' instead.') + def setModelTypeAsDenseNet(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('DenseNet121') + + @ deprecated(since='2.2.0', message='\'.setModelTypeAsMobileNetV2()\'' + ' has been deprecated! Please use' + ' \'set_model_type(\'MobileNetV2\')\' instead.') + def setModelTypeAsMobileNetV2(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('MobileNetV2') + + @ deprecated(since='2.2.0', message='\'.setModelTypeAsResNet50()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'ResNet50\')\' instead.') + def setModelTypeAsResNet50(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('ResNet50') + + @ deprecated(since='2.2.0', message='\'.setModelTypeAsDenseNet121()\'' + ' has been deprecated! Please use' + ' \'set_model_type(\'DenseNet121\')\' instead.') + def setModelTypeAsDenseNet121(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('DenseNet121') + + @ deprecated(since='2.2.0', message='\'.setModelTypeAsInceptionV3()\'' + ' has been deprecated! Please use' + ' \'set_model_type(\'InceptionV3\')\' instead.') + def setModelTypeAsInceptionV3(self): # pylint: disable=invalid-name + '''Function set in place to redirect users due to depreceation''' + return self.set_model_type('InceptionV3') diff --git a/imageai/Classification/__init__.py b/imageai/Classification/__init__.py index f02928e5..5f5676dd 100644 --- a/imageai/Classification/__init__.py +++ b/imageai/Classification/__init__.py @@ -1,3 +1,9 @@ +#!/usr/bin/python3 + +''' +Init file for image classification +''' + import tensorflow as tf from PIL import Image import numpy as np @@ -5,229 +11,215 @@ class ImageClassification: - """ - This is the image classification class in the ImageAI library. It provides support for 4 different models which are: - ResNet, MobileNetV2, DenseNet and Inception V3. After instantiating this class, you can set it's properties and - make image classification using it's pre-defined functions. - - The following functions are required to be called before a classification can be made - * setModelPath() - * At least of of the following and it must correspond to the model set in the setModelPath() - [setModelTypeAsMobileNetv2(), setModelTypeAsResNet(), setModelTypeAsDenseNet, setModelTypeAsInceptionV3] - * loadModel() [This must be called once only before making a classification] - - Once the above functions have been called, you can call the classifyImage() function of the classification instance - object at anytime to classify an image. - """ + ''' + This is the image classification class in the ImageAI library. It provides + support for 4 different models which are: ResNet, MobileNetV2, DenseNet + and Inception V3. After instantiating this class, you can set it's + properties and make image classification using it's pre-defined functions. + + The following functions are required to be called before a classification + can be made + * set_model_path() + * At least of of the following and it must correspond to the model set in + the set_model_path() + [setModelTypeAsMobileNetv2(), setModelTypeAsResNet(), + setModelTypeAsDenseNet, setModelTypeAsInceptionV3] + * load_model() [This must be called once only before making a + classification] + + Once the above functions have been called, you can call the + classify_image() function of the classification instance object at + anytime to classify an image. + ''' + def __init__(self): - self.__modelType = "" - self.modelPath = "" - self.__modelLoaded = False + self.model_path = '' + self.__model_type = '' + self.__model_loaded = False self.__model_collection = [] - self.__input_image_size = 224 - - def setModelPath(self, model_path): - """ - 'setModelPath()' function is required and is used to set the file path to the model adopted from the list of the - available 4 model types. The model path must correspond to the model type set for the classification instance object. + + def set_model_path(self, model_path): + ''' + 'set_model_path()' function is required and is used to set the file + path to the model adopted from the list of the available 4 model + types. The model path must correspond to the model type set for the + classification instance object. :param model_path: :return: - """ - self.modelPath = model_path + ''' + self.model_path = model_path + + def set_model_type(self, model_type): + ''' + This is a function that simply sets the model type. It was designed + with future compatibility, making it easy to switch out, update, + and remove models as they're updated and changed + + Take the model type input and make it lowercase to remove any + capitalizaton errors then set model type variable + ''' + match model_type.lower(): + case 'mobilenetv2': + self.__model_type = 'MobileNetV2' + case 'resnet50': + self.__model_type = 'ResNet50' + case 'densenet121': + self.__model_type = 'DenseNet121' + case 'inceptionv3': + self.__model_type = 'InceptionV3' def setModelTypeAsSqueezeNet(self): - raise ValueError("ImageAI no longer support SqueezeNet. You can use MobileNetV2 instead by downloading the MobileNetV2 model and call the function 'setModelTypeAsMobileNetV2'") - + raise ValueError( + 'ImageAI no longer support SqueezeNet. You can use MobileNetV2' + ' instead by downloading the MobileNetV2 model and call the' + ' function \'setModelTypeAsMobileNetV2\'') + + @deprecated(since='2.2.0', message='\'.setModelTypeAsMobileNetV2()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'MobileNetV2\')\' instead.') def setModelTypeAsMobileNetV2(self): - """ - 'setModelTypeAsMobileNetV2()' is used to set the model type to the MobileNetV2 model - for the classification instance object . - :return: - """ - self.__modelType = "mobilenetv2" - - @deprecated(since="2.1.6", message="'.setModelTypeAsResNet()' has been deprecated! Please use 'setModelTypeAsResNet50()' instead.") - def setModelTypeAsResNet(self): - return self.setModelTypeAsResNet50() + self.set_model_type('MobileNetV2') + @deprecated(since='2.2.0', message='\'.setModelTypeAsResNet50()\' has' + ' been deprecated! Please use' + ' \'set_model_type(\'ResNet50\')\' instead.') def setModelTypeAsResNet50(self): - """ - 'setModelTypeAsResNet50()' is used to set the model type to the ResNet50 model - for the classification instance object . - :return: - """ - self.__modelType = "resnet50" - - @deprecated(since="2.1.6", message="'.setModelTypeAsDenseNet()' has been deprecated! Please use 'setModelTypeAsDenseNet121()' instead.") - def setModelTypeAsDenseNet(self): - return self.setModelTypeAsDenseNet121() + self.set_model_type('ResNet50') + @deprecated(since='2.2.0', message='\'.setModelTypeAsDenseNet121()\'' + ' has been deprecated! Please use' + ' \'set_model_type(\'DenseNet121\')\' instead.') def setModelTypeAsDenseNet121(self): - """ - 'setModelTypeAsDenseNet121()' is used to set the model type to the DenseNet121 model - for the classification instance object . - :return: - """ - self.__modelType = "densenet121" + self.set_model_type('DenseNet121') + @deprecated(since='2.2.0', message='\'.setModelTypeAsInceptionV3()\'' + ' has been deprecated! Please use' + ' \'set_model_type(\'InceptionV3\')\' instead.') def setModelTypeAsInceptionV3(self): - """ - 'setModelTypeAsInceptionV3()' is used to set the model type to the InceptionV3 model - for the classification instance object . - :return: - """ - self.__modelType = "inceptionv3" - - def loadModel(self, classification_speed="normal"): - """ - 'loadModel()' function is used to load the model structure into the program from the file path defined - in the setModelPath() function. This function receives an optional value which is "classification_speed". - The value is used to reduce the time it takes to classify an image, down to about 50% of the normal time, - with just slight changes or drop in classification accuracy, depending on the nature of the image. - * classification_speed (optional); Acceptable values are "normal", "fast", "faster" and "fastest" + self.set_model_type('InceptionV3') + + def load_model(self): + ''' + 'load_model()' function is used to load the model structure into the + program from the file path defined in the set_model_path() function. + This function receives an optional value which is + 'classification_speed'. The value is used to reduce the time it takes + to classify an image, down to about 50% of the normal time, with just + slight changes or drop in classification accuracy, depending on the + nature of the image. + * classification_speed (optional); Acceptable values are 'normal', + 'fast', 'faster' and 'fastest' :param classification_speed : :return: - """ - - if(classification_speed=="normal"): - self.__input_image_size = 224 - elif(classification_speed=="fast"): - self.__input_image_size = 160 - elif(classification_speed=="faster"): - self.__input_image_size = 120 - elif (classification_speed == "fastest"): - self.__input_image_size = 100 - - if (self.__modelLoaded == False): - - if(self.__modelType == "" ): - raise ValueError("You must set a valid model type before loading the model.") - - elif(self.__modelType == "mobilenetv2"): - model = tf.keras.applications.MobileNetV2(input_shape=(self.__input_image_size, self.__input_image_size, 3), weights=None, classes = 1000 ) - model.load_weights(self.modelPath) - self.__model_collection.append(model) - self.__modelLoaded = True - try: - None - except: - raise ValueError("An error occured. Ensure your model file is a MobileNetV2 Model and is located in the path {}".format(self.modelPath)) - - elif(self.__modelType == "resnet50"): - try: - model = tf.keras.applications.ResNet50(input_shape=(self.__input_image_size, self.__input_image_size, 3), weights=None, classes = 1000 ) - model.load_weights(self.modelPath) - self.__model_collection.append(model) - self.__modelLoaded = True - except Exception as e: - raise ValueError("An error occured. Ensure your model file is a ResNet50 Model and is located in the path {}".format(self.modelPath)) - - elif (self.__modelType == "densenet121"): - try: - model = tf.keras.applications.DenseNet121(input_shape=(self.__input_image_size, self.__input_image_size, 3), weights=None, classes = 1000 ) - model.load_weights(self.modelPath) - self.__model_collection.append(model) - self.__modelLoaded = True - except: - raise ValueError("An error occured. Ensure your model file is a DenseNet121 Model and is located in the path {}".format(self.modelPath)) - - elif (self.__modelType == "inceptionv3"): - try: - model = tf.keras.applications.InceptionV3(input_shape=(self.__input_image_size, self.__input_image_size, 3), weights=None, classes = 1000 ) - model.load_weights(self.modelPath) - self.__model_collection.append(model) - self.__modelLoaded = True - except: - raise ValueError("An error occured. Ensure your model file is in {}".format(self.modelPath)) - - - def classifyImage(self, image_input, result_count=5, input_type="file"): - """ - 'classifyImage()' function is used to classify a given image by receiving the following arguments: - * input_type (optional) , the type of input to be parsed. Acceptable values are "file", "array" and "stream" - * image_input , file path/numpy array/image file stream of the image. - * result_count (optional) , the number of classifications to be sent which must be whole numbers between - 1 and 1000. The default is 5. - - This function returns 2 arrays namely 'classification_results' and 'classification_probabilities'. The 'classification_results' - contains possible objects classes arranged in descending of their percentage probabilities. The 'classification_probabilities' - contains the percentage probability of each object class. The position of each object class in the 'classification_results' - array corresponds with the positions of the percentage probability in the 'classification_probabilities' array. + ''' + + if not self.__model_loaded: + + if self.__model_type == '': + print('No model type specified, defaulting to ResNet50') + self.__model_type = 'ResNet50' + + match self.__model_type: + case 'MobileNetV2': + model = tf.keras.applications.mobilenet_v2.MobileNetV2() + case 'ResNet50': + model = tf.keras.applications.resnet50.ResNet50() + case 'DenseNet121': + model = tf.keras.applications.densenet.DenseNet121() + case 'InceptionV3': + model = tf.keras.applications.inception_v3.InceptionV3() + case _: + raise ValueError('You must set a valid model type before' + ' loading the model.') + + self.__model_collection.append(model) + self.__model_loaded = True + + def classify_image(self, image_input, result_count=5, input_type='file'): + ''' + 'classify_image()' function is used to classify a given image by + receiving the following arguments: + * input_type (optional) , the type of input to be parsed. + Acceptable values are 'file', 'array' and 'stream' + * image_input , file path/numpy array/image file stream of + the image. + * result_count (optional) , the number of classifications to + be sent which must be whole numbers between 1 and 1000. + The default is 5. + + This function returns 2 arrays namely 'classification_results' and + 'classification_probabilities'. The 'classification_results' contains + possible objects classes arranged in descending of their percentage + probabilities. The 'classification_probabilities' contains the + percentage probability of each object class. The position of each + object class in the 'classification_results' array corresponds with + the positions of the percentage probability in the + 'classification_probabilities' array. :param input_type: :param image_input: :param result_count: :return classification_results, classification_probabilities: - """ + ''' classification_results = [] classification_probabilities = [] - if (self.__modelLoaded == False): - raise ValueError("You must call the loadModel() function before making classification.") - - else: - if (input_type == "file"): - try: - image_to_predict = tf.keras.preprocessing.image.load_img(image_input, target_size=(self.__input_image_size, self.__input_image_size)) - image_to_predict = tf.keras.preprocessing.image.img_to_array(image_to_predict, data_format="channels_last") - image_to_predict = np.expand_dims(image_to_predict, axis=0) - except: - raise ValueError("You have set a path to an invalid image file.") - elif (input_type == "array"): - try: - image_input = Image.fromarray(np.uint8(image_input)) - image_input = image_input.resize((self.__input_image_size, self.__input_image_size)) - image_input = np.expand_dims(image_input, axis=0) - image_to_predict = image_input.copy() - image_to_predict = np.asarray(image_to_predict, dtype=np.float64) - except: - raise ValueError("You have parsed in a wrong numpy array for the image") - elif (input_type == "stream"): - try: - image_input = Image.open(image_input) - image_input = image_input.resize((self.__input_image_size, self.__input_image_size)) - image_input = np.expand_dims(image_input, axis=0) - image_to_predict = image_input.copy() - image_to_predict = np.asarray(image_to_predict, dtype=np.float64) - - except: - raise ValueError("You have parsed in a wrong stream for the image") - - if (self.__modelType == "mobilenetv2"): - image_to_predict = tf.keras.applications.mobilenet_v2.preprocess_input(image_to_predict) - elif (self.__modelType == "densenet121"): - image_to_predict = tf.keras.applications.densenet.preprocess_input(image_to_predict) - elif (self.__modelType == "inceptionv3"): - image_to_predict = tf.keras.applications.inception_v3.preprocess_input(image_to_predict) - - try: - model = self.__model_collection[0] - prediction = model.predict(image_to_predict, steps=1) - - if (self.__modelType == "mobilenetv2"): - predictiondata = tf.keras.applications.mobilenet_v2.decode_predictions(prediction, top=int(result_count)) - elif (self.__modelType == "resnet50"): - predictiondata = tf.keras.applications.resnet50.decode_predictions(prediction, top=int(result_count)) - elif (self.__modelType == "inceptionv3"): - predictiondata = tf.keras.applications.inception_v3.decode_predictions(prediction, top=int(result_count)) - elif (self.__modelType == "densenet121"): - predictiondata = tf.keras.applications.densenet.decode_predictions(prediction, top=int(result_count)) - - - - for results in predictiondata: - for result in results: - classification_results.append(str(result[1])) - classification_probabilities.append(result[2] * 100) - except: - raise ValueError("An error occured! Try again.") - - return classification_results, classification_probabilities - - - @deprecated(since="2.1.6", message="'.predictImage()' has been deprecated! Please use 'classifyImage()' instead.") - def predictImage(self, image_input, result_count=5, input_type="file"): - - return self.classifyImage(image_input, result_count, input_type) \ No newline at end of file + if not self.__model_loaded: + raise ValueError('You must call the load_model() function before' + ' making classification.') + + if input_type == 'file': + image_to_predict = tf.keras.utils\ + .load_img(image_input, target_size=( + 224, 224)) + image_to_predict = tf.keras.utils.img_to_array( + image_to_predict) + image_to_predict = tf.expand_dims(image_to_predict, 0) + elif input_type == 'array': + image_input = Image.fromarray(np.uint8(image_input)) + image_input = image_input.resize( + (224, 224)) + image_input = np.expand_dims(image_input, axis=0) + image_to_predict = image_input.copy() + image_to_predict = np.asarray( + image_to_predict, dtype=np.float64) + elif input_type == 'stream': + image_input = Image.open(image_input) + image_input = image_input.resize( + (224, 224)) + image_input = np.expand_dims(image_input, axis=0) + image_to_predict = image_input.copy() + image_to_predict = np.asarray( + image_to_predict, dtype=np.float64) + + app = None + match self.__model_type: + case 'MobileNetV2': + app = tf.keras.applications.mobilenet_v2 + case 'ResNet50': + app = tf.keras.applications.resnet50 + case 'DenseNet121': + app = tf.keras.applications.densenet + case 'InceptionV3': + app = tf.keras.applications.inception_v3 + image_to_predict = app.preprocess_input(image_to_predict) + + model = self.__model_collection[0] + prediction = model.predict(image_to_predict) + + prediction_data = app.decode_predictions(prediction, + top=int(result_count)) + + for results in prediction_data: + for result in results: + classification_results.append(str(result[1])) + classification_probabilities.append(result[2] * 100) + + return classification_results, classification_probabilities + + @deprecated(since='2.1.6', message='\'.predictImage()\' has been' + ' deprecated! Please use \'classify_image()\' instead.') + def predictImage(self, image_input, result_count=5, input_type='file'): + return self.classify_image(image_input, result_count, input_type) diff --git a/requirements.txt b/requirements.txt index 51fabd45..903813ac 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,9 +1,9 @@ -tensorflow==2.4.0 -keras==2.4.3 -numpy==1.19.3 -pillow==8.1.1 -scipy==1.4.1 -h5py==2.10.0 -matplotlib==3.3.2 -opencv-python +tensorflow==2.9.1 +keras==2.9.0 +numpy==1.23.1 +pillow==8.4.0 +scipy==1.9.0 +h5py==3.7.0 +matplotlib==3.5.2 +opencv-python==4.6.0.66 keras-resnet==0.2.0 diff --git a/setup.py b/setup.py index 9c6fec7e..a2add03b 100644 --- a/setup.py +++ b/setup.py @@ -1,14 +1,13 @@ from setuptools import setup,find_packages setup(name="imageai", - version='2.1.6', - description='A python library built to empower developers to build applications and systems with self-contained Computer Vision capabilities', + version='2.2.0', + description='A python library built to empower developers to build applications and systems with self-contained Computer Vision capabilities', url="https://github.com/OlafenwaMoses/ImageAI", author='Moses Olafenwa and John Olafenwa', author_email='guymodscientist@gmail.com', license='MIT', packages= find_packages(), - install_requires=['numpy==1.19.3','scipy==1.4.1','pillow==8.1.1',"matplotlib==3.3.2", "h5py==2.10.0", "keras-resnet==0.2.0", "opencv-python", "keras==2.4.3"], + install_requires=['tensorflow==2.9.1', 'keras==2.9.0', 'numpy==1.23.1', 'pillow==8.4.0', 'scipy==1.9.0', 'h5py==3.7.0', 'matplotlib==3.5.2', 'opencv-python==4.6.0.66', 'keras-resnet==0.2.0'], zip_safe=False - ) \ No newline at end of file