Using Artificial Neural Networks for Regression in Python

Blog, Case Studies-Python, Deep Learning / 49 Comments / By Farukh Hashmi

Artificial Neural Networks(ANN) can be used for a wide variety of tasks, from face recognition to self-driving cars to chatbots! To understand more about ANN in-depth please read this post and watch the below video!

ANN can be used for supervised ML regression problems as well.

In this post, I am going to show you how to implement a Deep Learning ANN for a Regression use case.

I am using the pre-processed data from a previous case study on predicting old car prices. You can check the data cleansing and feature selection steps there.

Data description

You can download the data pickle file required for this case study here.

The business meaning of each column in the data is as below

Price: The Price of the car in dollars
Age: The age of the car in months
KM: How many KMS did the car was used
HP: Horsepower of the car
MetColor: Whether the car has a metallic color or not
CC: The engine size of the car
Doors: The number of doors in the car
Weight: The weight of the car

Create an ML model which can predict the apt price of a second-hand car.

Defining the problem statement:

Target Variable: Price
Predictors: Age, KM, CC, etc.

Loading the data for regression

I am loading the preprocessed data ‘CarPricesData.pkl’. This data is the final list of features selected for ML.

# Reading the cleaned numeric car prices data
import pandas as pd
import numpy as np

# To remove the scientific notation from numpy arrays
np.set_printoptions(suppress=True)

CarPricesDataNumeric=pd.read_pickle('CarPricesData.pkl')
CarPricesDataNumeric.head()

# Reading the cleaned numeric car prices data

import pandas as pd

import numpy as np

# To remove the scientific notation from numpy arrays

np.set_printoptions(suppress=True)

CarPricesDataNumeric=pd.read_pickle('CarPricesData.pkl')

CarPricesDataNumeric.head()

Splitting the Data into Training and Testing

We don’t use the full data for creating the model. Some data is randomly selected and kept aside for checking how good the model is. This is known as Testing Data and the remaining data is called Training data on which the model is built. Typically 70% of data is used as Training data and the rest 30% is used as Testing data.

In this same step, we are standardizing the data as well. This is important for Neural Networks because it improves the model training speed and helps to find global minima.

# Separate Target Variable and Predictor Variables
TargetVariable=['Price']
Predictors=['Age', 'KM', 'Weight', 'HP', 'MetColor', 'CC', 'Doors']

X=CarPricesDataNumeric[Predictors].values
y=CarPricesDataNumeric[TargetVariable].values

### Sandardization of data ###
from sklearn.preprocessing import StandardScaler
PredictorScaler=StandardScaler()
TargetVarScaler=StandardScaler()

# Storing the fit object for later reference
PredictorScalerFit=PredictorScaler.fit(X)
TargetVarScalerFit=TargetVarScaler.fit(y)

# Generating the standardized values of X and y
X=PredictorScalerFit.transform(X)
y=TargetVarScalerFit.transform(y)

# Split the data into training and testing set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Quick sanity check with the shapes of Training and testing datasets
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)

# Separate Target Variable and Predictor Variables

TargetVariable=['Price']

Predictors=['Age', 'KM', 'Weight', 'HP', 'MetColor', 'CC', 'Doors']

X=CarPricesDataNumeric[Predictors].values

y=CarPricesDataNumeric[TargetVariable].values

### Sandardization of data ###

from sklearn.preprocessing import StandardScaler

PredictorScaler=StandardScaler()

TargetVarScaler=StandardScaler()

# Storing the fit object for later reference

PredictorScalerFit=PredictorScaler.fit(X)

TargetVarScalerFit=TargetVarScaler.fit(y)

# Generating the standardized values of X and y

X=PredictorScalerFit.transform(X)

y=TargetVarScalerFit.transform(y)

# Split the data into training and testing set

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Quick sanity check with the shapes of Training and testing datasets

print(X_train.shape)

print(y_train.shape)

print(X_test.shape)

print(y_test.shape)

Installing the required libraries

To implement deep learning ANNs, two libraries are required, ‘tensorflow‘ and ‘keras‘.

# Installing required libraries
!pip install tensorflow
!pip install keras

# Installing required libraries

!pip install tensorflow

!pip install keras

Creating Deep Learning- Artificial Neural Networks(ANN) model

The architecture of a Deep Learning ANN used in this case study is shown below

I am using two hidden layers with five neurons each and one output layer with one neuron. Can you change these numbers? Yes, you can change the number of hidden layers and the number of neurons in each layer.

Finally, choose the combination that produces the best possible accuracy. This is the process of tuning the ANN model.

ANN Architecture used in this case study

In the below code snippet, the “Sequential” module from the Keras library is used to create a sequence of ANN layers stacked one after the other. Each layer is defined using the “Dense” module of Keras where we specify how many neurons would be there, which technique would be used to initialize the weights in the network. what will be the activation function for each neuron in that layer etc

Lets quickly understand the hyperparameters in below code snippets

units=5: This means we are creating a layer with five neurons in it. Each of these five neurons will be receiving the values of inputs, for example, the values of ‘Age’ will be passed to all five neurons, similarly all other columns.
input_dim=7: This means there are seven predictors in the input data which is expected by the first layer. If you see the second dense layer, we don’t specify this value, because the Sequential model passes this information further to the next layers.
kernel_initializer=’normal’: When the Neurons start their computation, some algorithm has to decide the value for each weight. This parameter specifies that. You can choose different values for it like ‘normal’ or ‘glorot_uniform’.
activation=’relu’: This specifies the activation function for the calculations inside each neuron. You can choose values like ‘relu’, ‘tanh’, ‘sigmoid’, etc.
batch_size=20: This specifies how many rows will be passed to the Network in one go after which the SSE calculation will begin and the neural network will start adjusting its weights based on the errors.
When all the rows are passed in the batches of 20 rows each as specified in this parameter, then we call that 1-epoch. Or one full data cycle. This is also known as mini-batch gradient descent. A small value of batch_size will make the ANN look at the data slowly, like 2 rows at a time or 4 rows at a time which could lead to overfitting, as compared to a large value like 20 or 50 rows at a time, which will make the ANN look at the data fast which could lead to underfitting. Hence a proper value must be chosen using hyperparameter tuning.
Epochs=50: The same activity of adjusting weights continues for 50 times, as specified by this parameter. In simple terms, the ANN looks at the full training data 50 times and adjusts its weights.

To understand more about these calculations which happen inside a neuron, refer to this post.

# importing the libraries
from keras.models import Sequential
from keras.layers import Dense

# create ANN model
model = Sequential()

# Defining the Input layer and FIRST hidden layer, both are same!
model.add(Dense(units=5, input_dim=7, kernel_initializer='normal', activation='relu'))

# Defining the Second layer of the model
# after the first layer we don't have to specify input_dim as keras configure it automatically
model.add(Dense(units=5, kernel_initializer='normal', activation='tanh'))

# The output neuron is a single fully connected node 
# Since we will be predicting a single number
model.add(Dense(1, kernel_initializer='normal'))

# Compiling the model
model.compile(loss='mean_squared_error', optimizer='adam')

# Fitting the ANN to the Training set
model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)

# importing the libraries

from keras.models import Sequential

from keras.layers import Dense

# create ANN model

model = Sequential()

# Defining the Input layer and FIRST hidden layer, both are same!

model.add(Dense(units=5, input_dim=7, kernel_initializer='normal', activation='relu'))

# Defining the Second layer of the model

# after the first layer we don't have to specify input_dim as keras configure it automatically

model.add(Dense(units=5, kernel_initializer='normal', activation='tanh'))

# The output neuron is a single fully connected node

# Since we will be predicting a single number

model.add(Dense(1, kernel_initializer='normal'))

# Compiling the model

model.compile(loss='mean_squared_error', optimizer='adam')

# Fitting the ANN to the Training set

model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)

Hyperparameter tuning of ANN

Finding the best values for batch_size and epoch is very important as it directly affects the model performance. Bad values can lead to overfitting or underfitting. I am showing two approaches for tuning the parameters of the ANN. Apart from epoch and batch_size, you can also choose to tune the optimal number of neurons, the optimal number of layers, etc.

There is no thumb rule which can help you to decide the number of layers/number of neurons etc. in the first look at data. You need to try different parameters and choose the combination which produces the highest accuracy.

Just keep in mind, that, the bigger the network, the more computationally intensive it is, hence it will take more time to run. So always to find the best accuracy with the minimum number of layers/neurons.

Finding best set of parameters using manual grid search

This is a simple for loop based approach. You can easily edit this and adapt it for more hyperparameters by simply adding another nested for-loop.

# Defining a function to find the best parameters for ANN
def FunctionFindBestParams(X_train, y_train, X_test, y_test):
    
    # Defining the list of hyper parameters to try
    batch_size_list=[5, 10, 15, 20]
    epoch_list  =   [5, 10, 50, 100]
    
    import pandas as pd
    SearchResultsData=pd.DataFrame(columns=['TrialNumber', 'Parameters', 'Accuracy'])
    
    # initializing the trials
    TrialNumber=0
    for batch_size_trial in batch_size_list:
        for epochs_trial in epoch_list:
            TrialNumber+=1
            # create ANN model
            model = Sequential()
            # Defining the first layer of the model
            model.add(Dense(units=5, input_dim=X_train.shape[1], kernel_initializer='normal', activation='relu'))

            # Defining the Second layer of the model
            model.add(Dense(units=5, kernel_initializer='normal', activation='relu'))

            # The output neuron is a single fully connected node 
            # Since we will be predicting a single number
            model.add(Dense(1, kernel_initializer='normal'))

            # Compiling the model
            model.compile(loss='mean_squared_error', optimizer='adam')

            # Fitting the ANN to the Training set
            model.fit(X_train, y_train ,batch_size = batch_size_trial, epochs = epochs_trial, verbose=0)

            MAPE = np.mean(100 * (np.abs(y_test-model.predict(X_test))/y_test))
            
            # printing the results of the current iteration
            print(TrialNumber, 'Parameters:','batch_size:', batch_size_trial,'-', 'epochs:',epochs_trial, 'Accuracy:', 100-MAPE)
            
            SearchResultsData=SearchResultsData.append(pd.DataFrame(data=[[TrialNumber, str(batch_size_trial)+'-'+str(epochs_trial), 100-MAPE]],
                                                                    columns=['TrialNumber', 'Parameters', 'Accuracy'] ))
    return(SearchResultsData)


######################################################
# Calling the function
ResultsData=FunctionFindBestParams(X_train, y_train, X_test, y_test)

# Defining a function to find the best parameters for ANN

def FunctionFindBestParams(X_train, y_train, X_test, y_test):

# Defining the list of hyper parameters to try

batch_size_list=[5, 10, 15, 20]

epoch_list = [5, 10, 50, 100]

import pandas as pd

SearchResultsData=pd.DataFrame(columns=['TrialNumber', 'Parameters', 'Accuracy'])

# initializing the trials

TrialNumber=0

for batch_size_trial in batch_size_list:

for epochs_trial in epoch_list:

TrialNumber+=1

# create ANN model

model = Sequential()

# Defining the first layer of the model

model.add(Dense(units=5, input_dim=X_train.shape[1], kernel_initializer='normal', activation='relu'))

# Defining the Second layer of the model

model.add(Dense(units=5, kernel_initializer='normal', activation='relu'))

# The output neuron is a single fully connected node

# Since we will be predicting a single number

model.add(Dense(1, kernel_initializer='normal'))

# Compiling the model

model.compile(loss='mean_squared_error', optimizer='adam')

# Fitting the ANN to the Training set

model.fit(X_train, y_train ,batch_size = batch_size_trial, epochs = epochs_trial, verbose=0)

MAPE = np.mean(100 * (np.abs(y_test-model.predict(X_test))/y_test))

# printing the results of the current iteration

print(TrialNumber, 'Parameters:','batch_size:', batch_size_trial,'-', 'epochs:',epochs_trial, 'Accuracy:', 100-MAPE)

SearchResultsData=SearchResultsData.append(pd.DataFrame(data=[[TrialNumber, str(batch_size_trial)+'-'+str(epochs_trial), 100-MAPE]],

columns=['TrialNumber', 'Parameters', 'Accuracy'] ))

return(SearchResultsData)

######################################################

# Calling the function

ResultsData=FunctionFindBestParams(X_train, y_train, X_test, y_test)

Plotting the parameter trial results

%matplotlib inline
ResultsData.plot(x='Parameters', y='Accuracy', figsize=(15,4), kind='line')

1 2	%matplotlib inline ResultsData.plot(x='Parameters', y='Accuracy', figsize=(15,4), kind='line')

Visualizing the results of parameter trials for ANN

This graph shows that the best set of parameters are batch_size=15 and epochs=5. Next step is to train the model with these parameters.

Training the ANN model with the best parameters

Using the best set of parameters found above, training the model again and predicting the prices on testing data.

# Fitting the ANN to the Training set
model.fit(X_train, y_train ,batch_size = 15, epochs = 5, verbose=0)

# Generating Predictions on testing data
Predictions=model.predict(X_test)

# Scaling the predicted Price data back to original price scale
Predictions=TargetVarScalerFit.inverse_transform(Predictions)

# Scaling the y_test Price data back to original price scale
y_test_orig=TargetVarScalerFit.inverse_transform(y_test)

# Scaling the test data back to original scale
Test_Data=PredictorScalerFit.inverse_transform(X_test)

TestingData=pd.DataFrame(data=Test_Data, columns=Predictors)
TestingData['Price']=y_test_orig
TestingData['PredictedPrice']=Predictions
TestingData.head()

# Fitting the ANN to the Training set

model.fit(X_train, y_train ,batch_size = 15, epochs = 5, verbose=0)

# Generating Predictions on testing data

Predictions=model.predict(X_test)

# Scaling the predicted Price data back to original price scale

Predictions=TargetVarScalerFit.inverse_transform(Predictions)

# Scaling the y_test Price data back to original price scale

y_test_orig=TargetVarScalerFit.inverse_transform(y_test)

# Scaling the test data back to original scale

Test_Data=PredictorScalerFit.inverse_transform(X_test)

TestingData=pd.DataFrame(data=Test_Data, columns=Predictors)

TestingData['Price']=y_test_orig

TestingData['PredictedPrice']=Predictions

TestingData.head()

Finding the accuracy of the model

Using the final trained model, now we are generating the prediction error for each row in testing data as the Absolute Percentage Error. Taking the average for all the rows is known as Mean Absolute Percentage Error(MAPE).

The accuracy is calculated as 100-MAPE.

# Computing the absolute percent error
APE=100*(abs(TestingData['Price']-TestingData['PredictedPrice'])/TestingData['Price'])
TestingData['APE']=APE

print('The Accuracy of ANN model is:', 100-np.mean(APE))
TestingData.head()

# Computing the absolute percent error

APE=100*(abs(TestingData['Price']-TestingData['PredictedPrice'])/TestingData['Price'])

TestingData['APE']=APE

print('The Accuracy of ANN model is:', 100-np.mean(APE))

TestingData.head()

Error computation for the ANN predictions

Why the accuracy comes different every time I train ANN?

Even when you use the same hyperparameters, the result will be slightly different for each run of ANN. This happens because the initial step for ANN is the random initialization of weights. So every time you run the code, there are different values that get assigned to each neuron as weights and bias, hence the final outcome also differs slightly.

Finding best hyperparameters using GridSearchCV.

Apart from the manual search method shown above, you can also use the Grid Search Cross-validation method present in the sklearn library to find the best parameters of ANN.

The below snippet defines some parameter values to try and finds the best combination out of it.

# Function to generate Deep ANN model 
def make_regression_ann(Optimizer_trial):
    from keras.models import Sequential
    from keras.layers import Dense
    
    model = Sequential()
    model.add(Dense(units=5, input_dim=7, kernel_initializer='normal', activation='relu'))
    model.add(Dense(units=5, kernel_initializer='normal', activation='relu'))
    model.add(Dense(1, kernel_initializer='normal'))
    model.compile(loss='mean_squared_error', optimizer=Optimizer_trial)
    return model

###########################################
from sklearn.model_selection import GridSearchCV
from keras.wrappers.scikit_learn import KerasRegressor

# Listing all the parameters to try
Parameter_Trials={'batch_size':[10,20,30],
                      'epochs':[10,20],
                    'Optimizer_trial':['adam', 'rmsprop']
                 }

# Creating the regression ANN model
RegModel=KerasRegressor(make_regression_ann, verbose=0)

###########################################
from sklearn.metrics import make_scorer

# Defining a custom function to calculate accuracy
def Accuracy_Score(orig,pred):
    MAPE = np.mean(100 * (np.abs(orig-pred)/orig))
    print('#'*70,'Accuracy:', 100-MAPE)
    return(100-MAPE)

custom_Scoring=make_scorer(Accuracy_Score, greater_is_better=True)

#########################################
# Creating the Grid search space
# See different scoring methods by using sklearn.metrics.SCORERS.keys()
grid_search=GridSearchCV(estimator=RegModel, 
                         param_grid=Parameter_Trials, 
                         scoring=custom_Scoring, 
                         cv=5)

#########################################
# Measuring how much time it took to find the best params
import time
StartTime=time.time()

# Running Grid Search for different paramenters
grid_search.fit(X,y, verbose=1)

EndTime=time.time()
print("########## Total Time Taken: ", round((EndTime-StartTime)/60), 'Minutes')

print('### Printing Best parameters ###')
grid_search.best_params_

# Function to generate Deep ANN model

def make_regression_ann(Optimizer_trial):

from keras.models import Sequential

from keras.layers import Dense

model = Sequential()

model.add(Dense(units=5, input_dim=7, kernel_initializer='normal', activation='relu'))

model.add(Dense(units=5, kernel_initializer='normal', activation='relu'))

model.add(Dense(1, kernel_initializer='normal'))

model.compile(loss='mean_squared_error', optimizer=Optimizer_trial)

return model

###########################################

from sklearn.model_selection import GridSearchCV

from keras.wrappers.scikit_learn import KerasRegressor

# Listing all the parameters to try

Parameter_Trials={'batch_size':[10,20,30],

'epochs':[10,20],

'Optimizer_trial':['adam', 'rmsprop']

}

# Creating the regression ANN model

RegModel=KerasRegressor(make_regression_ann, verbose=0)

###########################################

from sklearn.metrics import make_scorer

# Defining a custom function to calculate accuracy

def Accuracy_Score(orig,pred):

MAPE = np.mean(100 * (np.abs(orig-pred)/orig))

print('#'*70,'Accuracy:', 100-MAPE)

return(100-MAPE)

custom_Scoring=make_scorer(Accuracy_Score, greater_is_better=True)

#########################################

# Creating the Grid search space

# See different scoring methods by using sklearn.metrics.SCORERS.keys()

grid_search=GridSearchCV(estimator=RegModel,

param_grid=Parameter_Trials,

scoring=custom_Scoring,

cv=5)

#########################################

# Measuring how much time it took to find the best params

import time

StartTime=time.time()

# Running Grid Search for different paramenters

grid_search.fit(X,y, verbose=1)

EndTime=time.time()

print("########## Total Time Taken: ", round((EndTime-StartTime)/60), 'Minutes')

print('### Printing Best parameters ###')

grid_search.best_params_

Results of hyperparameter search for ANN using GridSearch CV

Conclusion

This template can be used to fit the Deep Learning ANN regression model on any given dataset.

You can take the pre-processing steps of raw data from any of the case studies here.

Deep ANNs work great when you have a good amount of data available for learning. For small datasets with less than 50K records, I will recommend using the supervised ML models like Random Forests, Adaboosts, XGBoosts, etc.

The simple reason behind this is the high complexity and large computations of ANN. It is not worth it, if you can achieve the same accuracy with a faster and simpler model.

You look at deep learning ANNs only when you have a large amount of data available and the other algorithms are failing or do not fit for the task.

In the next post, I will show how to fit an ANN model for any classification dataset.

Author Details

Farukh Hashmi

Lead Data Scientist

Farukh is an innovator in solving industry problems using Artificial intelligence. His expertise is backed with 10 years of industry experience. Being a senior data scientist he is responsible for designing the AI/ML solution to provide maximum gains for the clients. As a thought leader, his focus is on solving the key business problems of the CPG Industry. He has worked across different domains like Telecom, Insurance, and Logistics. He has worked with global tech leaders including Infosys, IBM, and Persistent systems. His passion to teach inspired him to create this website!

https://thinkingneuron.com/

thinkingneuron@gmail.com

49 thoughts on “Using Artificial Neural Networks for Regression in Python”

Phakawat Lamchuan
May 25, 2021 at 4:54 am

Thank you very much for this example. I will try to adapt your example with my data. I am trying to predict salinity in the river to early warning for tap water production in Thailand.

Reply
1. Farukh Hashmi
  May 25, 2021 at 6:44 am
  
  Hi Phakawat!
  Very happy to see that this post has helped you in your work!
  Keep it up 🙂
  
  Reply
  1. Nino
    September 8, 2023 at 2:38 am
    
    Hi Farukh,
    Thanks very much for sharing your python codes and instructions for this case study how to use ANN for multiple variable regression.
    I’ve tried to follow your instructions and run your py code. I have some issue with declaring the input parameter matrix X as I found this errors:
    ” X=CarPricesDataNumeric[Predictors].values
    Traceback (most recent call last):
    File “”, line 1, in
    File “C:\Program Files\Python311\Lib\site-packages\pandas\core\frame.py”, line 3902, in __getitem__
    indexer = self.columns._get_indexer_strict(key, “columns”)[1]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Program Files\Python311\Lib\site-packages\pandas\core\indexes\base.py”, line 6114, in _get_indexer_strict
    self._raise_if_missing(keyarr, indexer, axis_name)
    File “C:\Program Files\Python311\Lib\site-packages\pandas\core\indexes\base.py”, line 6178, in _raise_if_missing
    raise KeyError(f”{not_found} not in index”)
    KeyError: “[‘Age’] not in index”
    
    Can you have how to fix this bug so that I could go on to test your ANN codes.
    
    Many thanks and kind regards,
    Nino Ng
    
    Reply
    1. Farukh Hashmi
      September 8, 2023 at 7:25 am
      
      Hi Nino,
      
      Thank you for kind words!
      It looks like Age column is missing from the input data. Can you just check if you have not dropped it in the previous steps
      
      Regards,
      Farukh Hashmi
      
      Reply
Katie
July 8, 2021 at 1:09 pm

Thank you for this its been really helpful!

Reply
1. Farukh Hashmi
  July 8, 2021 at 1:42 pm
  
  Hi Katie,
  
  Thank you for your kind words!
  I am happy that it was useful for you
  
  Reply
Ann
July 16, 2021 at 2:35 pm

MAPE = np.mean(100 * (np.abs(y_test-model.predict(X_test))/y_test))
the above formula can lead to negative error, so I recommend to use the following code,
from sklearn.metrics import mean_absolute_percentage_error
MAPE = mean_absolute_percentage_error(y_test, model.predict(X_test))

Reply
1. Farukh Hashmi
  July 16, 2021 at 3:12 pm
  
  Hi Ann,
  
  You are correct! Hence, whenever we are looking at regression results, we make sure to check median APE as well, as some of the predictions are bound to be bad, hence generating a large error and making the accuracy value negative. However, when a client looks into the dashboard, they tend to appreciate simple percentage differences instead of ML related computations like MAE, RMSE, etc. The MAPE implementation from sklearn does some massaging to the output and will require us to educate the users. That was the rationale behind evaluating the model in the same terms as the clients would see it using simple percentage differences between original and prediction. I hope that helps!
  
  Reply
  1. Andrea Lacros
    February 5, 2024 at 2:36 am
    
    Hi, thanks for the article. How would you handle predictors (columns) with categorical data? For example what if a column has countries and another one has sectors? Thanks!
    
    Reply
    1. Farukh Hashmi
      February 25, 2024 at 9:35 am
      
      Hi Andrea,
      
      We will have to treat categorical variables and convert them to numeric depending upon if its ordinal or nominal.
      See this case study for more details on this.
      https://thinkingneuron.com/restaurant-rating-prediction-case-study-in-python/
      
      Reply
Gill
September 16, 2021 at 11:03 am

Hi Farukh,
Thanks for sharing the article. I was trying it and got the following error:
cannot import name ‘mirrored_strategy_with_two_gpus_no_merge_call’ from ‘tensorflow.python.distribute.strategy_combinations’
I tried to search for some solutions on the internet but didn’t found any. Can you help with this?
I am using Python 3.9.7 (64 bit).
Thanks
Gill

Reply
1. Farukh Hashmi
  September 27, 2021 at 7:12 am
  
  Hi Gill,
  
  It is very difficult to debug code like this, if you can share the screenshot with full error, I might be able to help.
  
  Reply
Mohamad Darouich
November 1, 2021 at 3:45 pm

Hallo FARUKH, thank you for your effort.
i implemented this Tamplete on a Dataset its name “Housing_Price” to predict House prices , which i downloaded from Kaggle.
everything is ok but unfortunetly i become a zero accuracy when i train the Model but i dont really know what is the problem?!

i will be Thankfull if you could answer my question

Reply
1. Farukh Hashmi
  November 10, 2021 at 5:49 pm
  
  Hi Mohamad!
  
  It is difficult to comment about your code error without seeing your code! Can you share the screenshot of the cell and code where you are getting the accuracy as zero, maybe then I can help.
  
  Reply
pradeep yadav
November 24, 2021 at 5:24 pm

without cross-validation, the validation result compromised
how to use cross-validation, please suggest
Thanks!

Reply
1. Farukh Hashmi
  November 24, 2021 at 5:39 pm
  
  Hi Pradeep!
  
  You can use the GridSearchCV option. It produces the avg results based on k-Fold CV.
  
  Regards,
  Farukh Hashmi
  
  Reply
Rakesh P Rao
December 15, 2021 at 8:03 am

Mistake in the code… normalization should be done post train/test split. You have baised the test data.

Reply
1. Farukh Hashmi
  December 15, 2021 at 8:30 am
  
  Hi Rakesh,
  
  Thank you for your input!
  The testing data must be on the same level of encoding as of training data. Hence using the same fit object is necessary for standardisation/normalisation.
  If you do separate normalisation of test data, the range will vary and hence the predictions will be incorrect, because the model didn’t get the input at the same level it was trained on.
  Hope that helps!
  
  Regards,
  Farukh Hashmi
  
  Reply
Alice Liu
January 2, 2022 at 10:11 am

Thank you for your nice code,
However, I still don’t understand why and how are there 5 groups of results (0~4).

Reply
1. Farukh Hashmi
  January 2, 2022 at 10:28 am
  
  Hi Alice,
  
  If I am understanding it correct. You are referring to the cross validation step. Since we are using cv=5. The data is sampled 5-times hence, for each iteration there is one accuracy value.
  Hope that helps!
  
  Reply
Alice Liu
January 3, 2022 at 2:45 am

I mean the ANN results
Age
0 59.0 …
1 62.0 …
2 59.0 …
3 69.0 …
4 65.0 …
Thank you for your prompt reply,

Reply
1. vamraja
  November 30, 2023 at 4:32 pm
  
  Due to the usage of “.head()”, that prints only the first 5 rows. Similarly, adding “.tail()” fetches the final 5 rows. [.head() and .tail() by default fetch the first 5 and last 5 rows, but to fetch the first or last 5 columns, you need to mention “axis = 1” inside the bracket of head() or tail()]
  (:
  
  Reply
Arin
February 9, 2022 at 1:00 pm

HI Sir, Thank you for the guidance. One thing, can we use Hyperopt instead of gridsearch? Do you have any suggestions on that or a guide? Thank you

Reply
1. Farukh Hashmi
  February 9, 2022 at 1:24 pm
  
  Hi Arin,
  
  Yes you can use Hyperopt!
  Please read below article.
  https://thinkingneuron.com/how-to-tune-hyperparameters-automatically-using-bayesian-optimization/
  
  Reply
Sohail Iqbal
February 13, 2022 at 6:47 am

Dear Farukh Hashmi Thank you very much for your such a great effort. I apricate your efforts for this post and a comprehensive theoretical background in the video.

JazakaAllah Khair.

Reply
1. Farukh Hashmi
  February 13, 2022 at 7:04 am
  
  Thank you for the kind words Sohail!
  
  Reply
Sreyasi
April 18, 2022 at 5:56 am

Hi Farukh,

Thank you for the article but somehow I am unable to fit the X and Y to the grid search. After Epoch 1 runs successfully I get this error :
NotFittedError: All estimators failed to fit

Reply
1. Farukh Hashmi
  April 20, 2022 at 12:08 pm
  
  Hi Sreyasi!
  
  In the latest versions GridSearchCV is throwing such errors. Can you try Random SearchCV and see if it works?
  
  Reply
pranav
April 20, 2022 at 11:32 am

broo can u help me in doing a code..

Reply
Phakawat Lamchuan
May 18, 2022 at 9:48 am

Without your video and example, I will be unable to achieve my research. I have already published in Water, MDPI (https://doi.org/10.3390/w14050741). Thank you so much I really appreciate your guidance.

Reply
1. Farukh Hashmi
  May 18, 2022 at 9:53 am
  
  This is great news!!
  Keep up the great work. 🙂
  
  Reply
Mostafa
May 30, 2022 at 7:21 am

Hi Farukh,
Thank you so much for your efforts,
i have one question:
During the training process the loss = NAN
”
Epoch 1/50
67/67 [==============================] – 0s 1ms/step – loss: nan
Epoch 2/50
67/67 [==============================] – 0s 1ms/step – loss: nan”
Could you help me please?

Reply
1. Farukh Hashmi
  January 27, 2023 at 11:59 pm
  
  Hi Mostafa,
  
  Is this happening for all the runs?
  make sure your target variable does not contain any value that is zero. Because it may generate divide by zero issue or NANs
  
  Regards,
  Farukh Hashmi
  
  Reply
Sachdev
June 6, 2022 at 3:34 pm

hi Farukh! my data has already been split into training and testing data, what code should i write to reference these datas?

Reply
1. Farukh Hashmi
  January 27, 2023 at 11:57 pm
  
  Hi Sachdev,
  
  You can append train and test data to pass it for cross validation. Or simply train the model on train and validate the model on test data.
  
  Regards,
  Farukh Hashmi
  
  Reply
Mike
June 28, 2022 at 7:32 pm

Hi,
can you please tell me what kind of neural network this is? Is it a feedforward neural network or a CNN? Also can values of accuracy in the epoch calculating section surpass 100% and if so what does it mean for the model?

Reply
1. Farukh Hashmi
  January 27, 2023 at 11:55 pm
  
  Hi Mike!
  
  This is a Feed forward neural network.
  The accuracy should stay within 100%, However when you use custom metric like MAPE, then you may see negative results if there are outliers in the errors.
  
  Regards,
  Farukh Hashmi
  
  Reply
Zubiya Rao
December 1, 2022 at 11:29 am

Would you please suggest me how to select number of units for each layer?
In my data input_dim=3.

Reply
Godswill William
January 15, 2023 at 8:28 am

Very helpful tutorial, please can you help with the neural network equation..?

Reply
1. Farukh Hashmi
  January 27, 2023 at 11:52 pm
  
  Hi Godswill!
  I have tried to explain the equation of a Neuron in the video.
  https://youtu.be/i3crGbdElSw
  Hope that helps!
  
  Regards,
  Farukh Hashmi
  
  Reply
Soumyajyoti Kabi
January 24, 2023 at 8:33 am

Very helpful post. After checking several website, lastly I succeed to fit my data with the given code. Can you help me regarding tuning the number of neurons and layers?

Reply
1. Farukh Hashmi
  January 27, 2023 at 11:45 pm
  
  Hi Soumya,
  You start with a smaller number for each and then keep increasing them and note accuracy after every change.
  The idea is to find that sweet spot where you are getting highest accuracy with minimum number of neurons and layers. Because after a point you will see the testing accuracy decrease, that means model has started overfitting.
  
  Always remember, every neuron you add is increasing one more equation for the overall model!
  
  Hope this helps!
  
  Regards,
  Farukh Hashmi
  
  Reply
Sam
January 27, 2023 at 12:19 am

Dear Farukh Hashmi, Thank you very much for your such a great article. I have a follow up question. Once you have the final best model ready, how do we apply it to a new dataset to predict the second hand car price?

Reply
1. Farukh Hashmi
  January 27, 2023 at 11:40 pm
  
  Hi
  Thank you for your kind words!
  Please see this post and look at the deployment section.
  
  https://thinkingneuron.com/car-price-prediction-case-study-in-python/
  
  Regards
  Farukh Hashmi
  
  Reply
Don-tsa Delchere
February 23, 2023 at 11:49 am

Thank you very much for this tutorial.

but I am confused. I have read in different place that the accuracy is not a good metric for regression. can you please help me to understand while you used accuracy?

Please also I would like to have an example using may be mean squared error or determination coefficient. also the kind of plot I can do to confirm my result.

thank you and sorry for too much questions

Reply
Yonas Tilahun
April 24, 2023 at 2:11 pm

greeting all I got this code very helpful . can you tell how to get coefficients’ to write an empirical formula. how can I extend this in getting weight and biases.

Reply
Anila Joy
January 12, 2024 at 8:18 am

Dear Farukh Hashmi,
Thank you for the article and video. It was very helpful.

May you please advise the reason for getting accuracy above 100?
2/2 [==============================] – 0s 0s/step
9 Parameters: batch_size: 15 – epochs: 5 Accuracy: 128.60392062417344
2/2 [==============================] – 0s 0s/step
5 Parameters: batch_size: 10 – epochs: 5 Accuracy: 137.93475685949863

Reply
Jeff
July 31, 2024 at 4:15 pm

Hi! I am getting an error when I run this that says “incompatible shapes: [##] vs. [##,%%]” (the ## is the same number, and the %% is a different number). Do you have any idea why this might be happening or how I can fix it?

Reply
Nadji
November 26, 2024 at 6:29 pm

Hi Farukh,
Your work is helping me a lot—many thanks. However, could you please help me on this issue?
I am trying to move from Matlab to Python, and I was wondering why you only split between
Train and Test. In Matlab, we are used to splitting between Train, Validate, and Test. The Validation
split helps to fine-tune the internal hyperparameters as I am sure you know. Why we cannot see this
in your Python Programs ? and if in the case it wasn’t necessary, can it be done and how ?
Many Thanks Again.

Reply

Data description

Defining the problem statement:

Loading the data for regression

Splitting the Data into Training and Testing

Installing the required libraries

Creating Deep Learning- Artificial Neural Networks(ANN) model

Hyperparameter tuning of ANN

Finding best set of parameters using manual grid search

Plotting the parameter trial results

Training the ANN model with the best parameters

Finding the accuracy of the model

Why the accuracy comes different every time I train ANN?

Finding best hyperparameters using GridSearchCV.

Conclusion

49 thoughts on “Using Artificial Neural Networks for Regression in Python”

Leave a Reply! Cancel Reply