Optimization#
In The Mean and Slopes, we used a simple but slow way to find the slope that best predicted one vector of values from another vector of values.
First we go back to find that slope.
import numpy as np
import matplotlib.pyplot as plt
# Make plots look a little bit more fancy
plt.style.use('fivethirtyeight')
# Print to 2 decimal places, show tiny values as 0
np.set_printoptions(precision=2, suppress=True)
import pandas as pd
pd.set_option('mode.copy_on_write', True)
Download the ckd_clean.csv file to the
same directory as this notebook, if you are running on your own computer.
We fetch and process the data. See mean and slopes for a slower description of this processing.
# Load the data file
ckd = pd.read_csv('ckd_clean.csv')
pcv = np.array(ckd['Packed Cell Volume'])
hgb = np.array(ckd['Hemoglobin'])
Our criterion is the root mean squared error (RMSE).
def calc_rmse(slope):
predicted_pcv = hgb * slope # 'hgb' comes from the top level
errors = pcv - predicted_pcv # 'pcv' comes from the top level
return np.sqrt(np.mean(errors ** 2))
We found the best slope by trying a very large number of slopes, and recording, for each slope, the root mean squared error. We chose the slope from the slopes that we tried, that gave us the lowest root mean squared error.
# Slopes to try
some_slopes = np.arange(2, 4, 0.01)
n_slopes = len(some_slopes)
# Try all these slopes, calculate and record RMSE.
rmse_errors = np.zeros(n_slopes)
best_slope = None
least_error = np.inf
for i in np.arange(n_slopes):
# Get the slope.
slope = some_slopes[i]
# Calculate the error measure.
this_error = calc_rmse(slope)
# Record this slope if this is the smallest error.
if this_error < least_error:
least_error = this_error
best_slope = slope
# Put the error into the array of errors
rmse_errors[i] = this_error
# The slope minimizing the RMSE.
best_slope
3.0499999999999776
At the end, of the mean and slopes notebook, we saw that
a function in Scipy called minimize can do this work for us, relatively
quickly.
from scipy.optimize import minimize
minimize(calc_rmse, 3)
message: Optimization terminated successfully.
success: True
status: 0
fun: 4.786330607301848
x: [ 3.047e+00]
nit: 3
jac: [ 2.384e-07]
hess_inv: [[ 2.447e-02]]
nfev: 10
njev: 5
What we are doing, with our slow dumb technique, and with the minimize
function, is something called mathematical
optimization. We use
optimization when we have some function that takes one or more
parameters. We want to chose, or optimize the parameters to give us some
desired output from the function.
In our case our function is the root mean squared error, calc_rmse. The
parameter is the slope. We are trying to find the value for the parameter
that minimizes the result of calling the function calc_rmse.
One way of doing this minimization, is the slow dumb way. We just try a huge number of values for the parameter (the slope), and chose the value that gives us the lowest output value (the root mean squared error).
This is such a common problem, that there has been an enormous amount of theoretical and practical work on building algorithms to make process of searching for the minimum value more efficient.
This notebook is to give you an idea of how you might do this, and therefore,
the kind of things that minimize can do, to search quickly for the best
parameter.
Let’s look again at the shape of the curve relating the slope to the root mean squared error:
plt.plot(some_slopes, rmse_errors)
plt.xlabel('Candidate slopes')
plt.ylabel('Root mean squared error')
plt.title('RMSE as a function of slope')
Text(0.5, 1.0, 'RMSE as a function of slope')
This is the function we are trying minimize. Specifically, we are trying to optimize the function that gives the SSE as a function of the slope parameter.
We want to avoid trying every possible value for the slope.
To do this, we are going to start with one value for the slope, say 100, then see if there is a good way to chose the next value to try.
Looking at the graph, we see that, when the slope is far away from the minimum, the RMSE (on the y axis) changes very quickly as the slope changes. That is, the function has a steep gradient.
Maybe we could check what the gradient is, at our starting value of 100, by calculating the RMSE (y) value, and then calculating the RMSE (y) value when we increase the slope by a tiny amount. This is the change in y for a very small change in x. We divide the change in y by the change in x, to get the gradient:
def calc_rmse_gradient(x, dx=0.0001):
# Gradient of the RMSE at this value of x
# RMSE at this x value.
sos_0 = calc_rmse(x)
# RMSE a tiny bit to the right on the x axis.
sos_1 = calc_rmse(x + dx)
# gradient is y difference divided by x difference.
return (sos_1 - sos_0) / dx
# The y value of the function.
calc_rmse(4)
14.155099273514049
# The gradient of the function at this point.
calc_rmse_gradient(4)
13.161923464508618
A large positive gradient means the x value (slope) that we tried is still far to the right of the minimum. This might encourage us to try an x value that is well to the left. We could call this a large step in x, and therefore a large step size.
Let’s try another value:
# The y value of the function.
calc_rmse(2)
15.411987772140478
# The gradient of the function at this point.
calc_rmse_gradient(2)
-13.294037130133773
A large negative gradient means the x value (slope) that we tried is still far to the left of the minimum. This might encourage us to try an x value that is well to the right.
As the gradients get small, we want to take smaller steps, so we don’t miss the minimum.
The general idea then, is to chose our step sizes in proportion to the gradient of the function.
This is the optimization technique known as gradient descent.
Here it is in action. We try new x-axis values by making big jumps when the gradient is steep, and small jumps when the gradient is shallow. For each new value of x, we check the gradient, and choose our new jump size.
Remember each value on the x-axis is another value we want to try, in order to find the best-fitting slope for our original problem.
next_x = 4 # We start the search at x=4
gamma = 0.00001 # Step size multiplier
precision = 0.00001 # Desired precision of result
max_iters = 1000 # Maximum number of iterations
for i in np.arange(max_iters):
# Go to the next x value
current_x = next_x
# Estimate the gradient
gradient = calc_rmse_gradient(current_x)
# Use gradient to choose the next x value to try.
# This takes negative steps when the gradient is positive
# and positive steps when the gradient is negative.
next_x = current_x - gamma * gradient
step = next_x - current_x
print('x: {:0.5f}; step {:0.5f}; gradient {:0.2f}'.format(
current_x, step, gradient))
# When the step size is equal to or smaller than the desired
# precision, we are near enough.
if abs(step) <= precision:
# Break out of the loop.
break
print("Minimum at", next_x)
x: 4.00000; step -0.00013; gradient 13.16
x: 3.99987; step -0.00013; gradient 13.16
x: 3.99974; step -0.00013; gradient 13.16
x: 3.99961; step -0.00013; gradient 13.16
x: 3.99947; step -0.00013; gradient 13.16
x: 3.99934; step -0.00013; gradient 13.16
x: 3.99921; step -0.00013; gradient 13.16
x: 3.99908; step -0.00013; gradient 13.16
x: 3.99895; step -0.00013; gradient 13.16
x: 3.99882; step -0.00013; gradient 13.16
x: 3.99868; step -0.00013; gradient 13.16
x: 3.99855; step -0.00013; gradient 13.16
x: 3.99842; step -0.00013; gradient 13.16
x: 3.99829; step -0.00013; gradient 13.16
x: 3.99816; step -0.00013; gradient 13.16
x: 3.99803; step -0.00013; gradient 13.16
x: 3.99789; step -0.00013; gradient 13.16
x: 3.99776; step -0.00013; gradient 13.16
x: 3.99763; step -0.00013; gradient 13.16
x: 3.99750; step -0.00013; gradient 13.16
x: 3.99737; step -0.00013; gradient 13.16
x: 3.99724; step -0.00013; gradient 13.16
x: 3.99710; step -0.00013; gradient 13.16
x: 3.99697; step -0.00013; gradient 13.16
x: 3.99684; step -0.00013; gradient 13.16
x: 3.99671; step -0.00013; gradient 13.16
x: 3.99658; step -0.00013; gradient 13.16
x: 3.99645; step -0.00013; gradient 13.16
x: 3.99632; step -0.00013; gradient 13.16
x: 3.99618; step -0.00013; gradient 13.16
x: 3.99605; step -0.00013; gradient 13.16
x: 3.99592; step -0.00013; gradient 13.16
x: 3.99579; step -0.00013; gradient 13.16
x: 3.99566; step -0.00013; gradient 13.16
x: 3.99553; step -0.00013; gradient 13.15
x: 3.99539; step -0.00013; gradient 13.15
x: 3.99526; step -0.00013; gradient 13.15
x: 3.99513; step -0.00013; gradient 13.15
x: 3.99500; step -0.00013; gradient 13.15
x: 3.99487; step -0.00013; gradient 13.15
x: 3.99474; step -0.00013; gradient 13.15
x: 3.99461; step -0.00013; gradient 13.15
x: 3.99447; step -0.00013; gradient 13.15
x: 3.99434; step -0.00013; gradient 13.15
x: 3.99421; step -0.00013; gradient 13.15
x: 3.99408; step -0.00013; gradient 13.15
x: 3.99395; step -0.00013; gradient 13.15
x: 3.99382; step -0.00013; gradient 13.15
x: 3.99368; step -0.00013; gradient 13.15
x: 3.99355; step -0.00013; gradient 13.15
x: 3.99342; step -0.00013; gradient 13.15
x: 3.99329; step -0.00013; gradient 13.15
x: 3.99316; step -0.00013; gradient 13.15
x: 3.99303; step -0.00013; gradient 13.15
x: 3.99290; step -0.00013; gradient 13.15
x: 3.99276; step -0.00013; gradient 13.15
x: 3.99263; step -0.00013; gradient 13.15
x: 3.99250; step -0.00013; gradient 13.15
x: 3.99237; step -0.00013; gradient 13.15
x: 3.99224; step -0.00013; gradient 13.15
x: 3.99211; step -0.00013; gradient 13.15
x: 3.99198; step -0.00013; gradient 13.15
x: 3.99184; step -0.00013; gradient 13.15
x: 3.99171; step -0.00013; gradient 13.15
x: 3.99158; step -0.00013; gradient 13.15
x: 3.99145; step -0.00013; gradient 13.15
x: 3.99132; step -0.00013; gradient 13.15
x: 3.99119; step -0.00013; gradient 13.15
x: 3.99105; step -0.00013; gradient 13.15
x: 3.99092; step -0.00013; gradient 13.15
x: 3.99079; step -0.00013; gradient 13.15
x: 3.99066; step -0.00013; gradient 13.15
x: 3.99053; step -0.00013; gradient 13.15
x: 3.99040; step -0.00013; gradient 13.15
x: 3.99027; step -0.00013; gradient 13.15
x: 3.99013; step -0.00013; gradient 13.15
x: 3.99000; step -0.00013; gradient 13.15
x: 3.98987; step -0.00013; gradient 13.15
x: 3.98974; step -0.00013; gradient 13.15
x: 3.98961; step -0.00013; gradient 13.15
x: 3.98948; step -0.00013; gradient 13.15
x: 3.98935; step -0.00013; gradient 13.14
x: 3.98921; step -0.00013; gradient 13.14
x: 3.98908; step -0.00013; gradient 13.14
x: 3.98895; step -0.00013; gradient 13.14
x: 3.98882; step -0.00013; gradient 13.14
x: 3.98869; step -0.00013; gradient 13.14
x: 3.98856; step -0.00013; gradient 13.14
x: 3.98843; step -0.00013; gradient 13.14
x: 3.98829; step -0.00013; gradient 13.14
x: 3.98816; step -0.00013; gradient 13.14
x: 3.98803; step -0.00013; gradient 13.14
x: 3.98790; step -0.00013; gradient 13.14
x: 3.98777; step -0.00013; gradient 13.14
x: 3.98764; step -0.00013; gradient 13.14
x: 3.98751; step -0.00013; gradient 13.14
x: 3.98737; step -0.00013; gradient 13.14
x: 3.98724; step -0.00013; gradient 13.14
x: 3.98711; step -0.00013; gradient 13.14
x: 3.98698; step -0.00013; gradient 13.14
x: 3.98685; step -0.00013; gradient 13.14
x: 3.98672; step -0.00013; gradient 13.14
x: 3.98659; step -0.00013; gradient 13.14
x: 3.98645; step -0.00013; gradient 13.14
x: 3.98632; step -0.00013; gradient 13.14
x: 3.98619; step -0.00013; gradient 13.14
x: 3.98606; step -0.00013; gradient 13.14
x: 3.98593; step -0.00013; gradient 13.14
x: 3.98580; step -0.00013; gradient 13.14
x: 3.98567; step -0.00013; gradient 13.14
x: 3.98553; step -0.00013; gradient 13.14
x: 3.98540; step -0.00013; gradient 13.14
x: 3.98527; step -0.00013; gradient 13.14
x: 3.98514; step -0.00013; gradient 13.14
x: 3.98501; step -0.00013; gradient 13.14
x: 3.98488; step -0.00013; gradient 13.14
x: 3.98475; step -0.00013; gradient 13.14
x: 3.98461; step -0.00013; gradient 13.14
x: 3.98448; step -0.00013; gradient 13.14
x: 3.98435; step -0.00013; gradient 13.14
x: 3.98422; step -0.00013; gradient 13.14
x: 3.98409; step -0.00013; gradient 13.14
x: 3.98396; step -0.00013; gradient 13.14
x: 3.98383; step -0.00013; gradient 13.14
x: 3.98370; step -0.00013; gradient 13.14
x: 3.98356; step -0.00013; gradient 13.14
x: 3.98343; step -0.00013; gradient 13.14
x: 3.98330; step -0.00013; gradient 13.13
x: 3.98317; step -0.00013; gradient 13.13
x: 3.98304; step -0.00013; gradient 13.13
x: 3.98291; step -0.00013; gradient 13.13
x: 3.98278; step -0.00013; gradient 13.13
x: 3.98264; step -0.00013; gradient 13.13
x: 3.98251; step -0.00013; gradient 13.13
x: 3.98238; step -0.00013; gradient 13.13
x: 3.98225; step -0.00013; gradient 13.13
x: 3.98212; step -0.00013; gradient 13.13
x: 3.98199; step -0.00013; gradient 13.13
x: 3.98186; step -0.00013; gradient 13.13
x: 3.98173; step -0.00013; gradient 13.13
x: 3.98159; step -0.00013; gradient 13.13
x: 3.98146; step -0.00013; gradient 13.13
x: 3.98133; step -0.00013; gradient 13.13
x: 3.98120; step -0.00013; gradient 13.13
x: 3.98107; step -0.00013; gradient 13.13
x: 3.98094; step -0.00013; gradient 13.13
x: 3.98081; step -0.00013; gradient 13.13
x: 3.98067; step -0.00013; gradient 13.13
x: 3.98054; step -0.00013; gradient 13.13
x: 3.98041; step -0.00013; gradient 13.13
x: 3.98028; step -0.00013; gradient 13.13
x: 3.98015; step -0.00013; gradient 13.13
x: 3.98002; step -0.00013; gradient 13.13
x: 3.97989; step -0.00013; gradient 13.13
x: 3.97976; step -0.00013; gradient 13.13
x: 3.97962; step -0.00013; gradient 13.13
x: 3.97949; step -0.00013; gradient 13.13
x: 3.97936; step -0.00013; gradient 13.13
x: 3.97923; step -0.00013; gradient 13.13
x: 3.97910; step -0.00013; gradient 13.13
x: 3.97897; step -0.00013; gradient 13.13
x: 3.97884; step -0.00013; gradient 13.13
x: 3.97871; step -0.00013; gradient 13.13
x: 3.97857; step -0.00013; gradient 13.13
x: 3.97844; step -0.00013; gradient 13.13
x: 3.97831; step -0.00013; gradient 13.13
x: 3.97818; step -0.00013; gradient 13.13
x: 3.97805; step -0.00013; gradient 13.13
x: 3.97792; step -0.00013; gradient 13.13
x: 3.97779; step -0.00013; gradient 13.13
x: 3.97766; step -0.00013; gradient 13.13
x: 3.97752; step -0.00013; gradient 13.13
x: 3.97739; step -0.00013; gradient 13.13
x: 3.97726; step -0.00013; gradient 13.12
x: 3.97713; step -0.00013; gradient 13.12
x: 3.97700; step -0.00013; gradient 13.12
x: 3.97687; step -0.00013; gradient 13.12
x: 3.97674; step -0.00013; gradient 13.12
x: 3.97661; step -0.00013; gradient 13.12
x: 3.97647; step -0.00013; gradient 13.12
x: 3.97634; step -0.00013; gradient 13.12
x: 3.97621; step -0.00013; gradient 13.12
x: 3.97608; step -0.00013; gradient 13.12
x: 3.97595; step -0.00013; gradient 13.12
x: 3.97582; step -0.00013; gradient 13.12
x: 3.97569; step -0.00013; gradient 13.12
x: 3.97556; step -0.00013; gradient 13.12
x: 3.97542; step -0.00013; gradient 13.12
x: 3.97529; step -0.00013; gradient 13.12
x: 3.97516; step -0.00013; gradient 13.12
x: 3.97503; step -0.00013; gradient 13.12
x: 3.97490; step -0.00013; gradient 13.12
x: 3.97477; step -0.00013; gradient 13.12
x: 3.97464; step -0.00013; gradient 13.12
x: 3.97451; step -0.00013; gradient 13.12
x: 3.97437; step -0.00013; gradient 13.12
x: 3.97424; step -0.00013; gradient 13.12
x: 3.97411; step -0.00013; gradient 13.12
x: 3.97398; step -0.00013; gradient 13.12
x: 3.97385; step -0.00013; gradient 13.12
x: 3.97372; step -0.00013; gradient 13.12
x: 3.97359; step -0.00013; gradient 13.12
x: 3.97346; step -0.00013; gradient 13.12
x: 3.97332; step -0.00013; gradient 13.12
x: 3.97319; step -0.00013; gradient 13.12
x: 3.97306; step -0.00013; gradient 13.12
x: 3.97293; step -0.00013; gradient 13.12
x: 3.97280; step -0.00013; gradient 13.12
x: 3.97267; step -0.00013; gradient 13.12
x: 3.97254; step -0.00013; gradient 13.12
x: 3.97241; step -0.00013; gradient 13.12
x: 3.97228; step -0.00013; gradient 13.12
x: 3.97214; step -0.00013; gradient 13.12
x: 3.97201; step -0.00013; gradient 13.12
x: 3.97188; step -0.00013; gradient 13.12
x: 3.97175; step -0.00013; gradient 13.12
x: 3.97162; step -0.00013; gradient 13.12
x: 3.97149; step -0.00013; gradient 13.12
x: 3.97136; step -0.00013; gradient 13.11
x: 3.97123; step -0.00013; gradient 13.11
x: 3.97110; step -0.00013; gradient 13.11
x: 3.97096; step -0.00013; gradient 13.11
x: 3.97083; step -0.00013; gradient 13.11
x: 3.97070; step -0.00013; gradient 13.11
x: 3.97057; step -0.00013; gradient 13.11
x: 3.97044; step -0.00013; gradient 13.11
x: 3.97031; step -0.00013; gradient 13.11
x: 3.97018; step -0.00013; gradient 13.11
x: 3.97005; step -0.00013; gradient 13.11
x: 3.96991; step -0.00013; gradient 13.11
x: 3.96978; step -0.00013; gradient 13.11
x: 3.96965; step -0.00013; gradient 13.11
x: 3.96952; step -0.00013; gradient 13.11
x: 3.96939; step -0.00013; gradient 13.11
x: 3.96926; step -0.00013; gradient 13.11
x: 3.96913; step -0.00013; gradient 13.11
x: 3.96900; step -0.00013; gradient 13.11
x: 3.96887; step -0.00013; gradient 13.11
x: 3.96873; step -0.00013; gradient 13.11
x: 3.96860; step -0.00013; gradient 13.11
x: 3.96847; step -0.00013; gradient 13.11
x: 3.96834; step -0.00013; gradient 13.11
x: 3.96821; step -0.00013; gradient 13.11
x: 3.96808; step -0.00013; gradient 13.11
x: 3.96795; step -0.00013; gradient 13.11
x: 3.96782; step -0.00013; gradient 13.11
x: 3.96769; step -0.00013; gradient 13.11
x: 3.96756; step -0.00013; gradient 13.11
x: 3.96742; step -0.00013; gradient 13.11
x: 3.96729; step -0.00013; gradient 13.11
x: 3.96716; step -0.00013; gradient 13.11
x: 3.96703; step -0.00013; gradient 13.11
x: 3.96690; step -0.00013; gradient 13.11
x: 3.96677; step -0.00013; gradient 13.11
x: 3.96664; step -0.00013; gradient 13.11
x: 3.96651; step -0.00013; gradient 13.11
x: 3.96638; step -0.00013; gradient 13.11
x: 3.96624; step -0.00013; gradient 13.11
x: 3.96611; step -0.00013; gradient 13.11
x: 3.96598; step -0.00013; gradient 13.11
x: 3.96585; step -0.00013; gradient 13.11
x: 3.96572; step -0.00013; gradient 13.11
x: 3.96559; step -0.00013; gradient 13.10
x: 3.96546; step -0.00013; gradient 13.10
x: 3.96533; step -0.00013; gradient 13.10
x: 3.96520; step -0.00013; gradient 13.10
x: 3.96506; step -0.00013; gradient 13.10
x: 3.96493; step -0.00013; gradient 13.10
x: 3.96480; step -0.00013; gradient 13.10
x: 3.96467; step -0.00013; gradient 13.10
x: 3.96454; step -0.00013; gradient 13.10
x: 3.96441; step -0.00013; gradient 13.10
x: 3.96428; step -0.00013; gradient 13.10
x: 3.96415; step -0.00013; gradient 13.10
x: 3.96402; step -0.00013; gradient 13.10
x: 3.96389; step -0.00013; gradient 13.10
x: 3.96375; step -0.00013; gradient 13.10
x: 3.96362; step -0.00013; gradient 13.10
x: 3.96349; step -0.00013; gradient 13.10
x: 3.96336; step -0.00013; gradient 13.10
x: 3.96323; step -0.00013; gradient 13.10
x: 3.96310; step -0.00013; gradient 13.10
x: 3.96297; step -0.00013; gradient 13.10
x: 3.96284; step -0.00013; gradient 13.10
x: 3.96271; step -0.00013; gradient 13.10
x: 3.96258; step -0.00013; gradient 13.10
x: 3.96244; step -0.00013; gradient 13.10
x: 3.96231; step -0.00013; gradient 13.10
x: 3.96218; step -0.00013; gradient 13.10
x: 3.96205; step -0.00013; gradient 13.10
x: 3.96192; step -0.00013; gradient 13.10
x: 3.96179; step -0.00013; gradient 13.10
x: 3.96166; step -0.00013; gradient 13.10
x: 3.96153; step -0.00013; gradient 13.10
x: 3.96140; step -0.00013; gradient 13.10
x: 3.96127; step -0.00013; gradient 13.10
x: 3.96113; step -0.00013; gradient 13.10
x: 3.96100; step -0.00013; gradient 13.10
x: 3.96087; step -0.00013; gradient 13.10
x: 3.96074; step -0.00013; gradient 13.10
x: 3.96061; step -0.00013; gradient 13.10
x: 3.96048; step -0.00013; gradient 13.10
x: 3.96035; step -0.00013; gradient 13.10
x: 3.96022; step -0.00013; gradient 13.10
x: 3.96009; step -0.00013; gradient 13.10
x: 3.95996; step -0.00013; gradient 13.09
x: 3.95983; step -0.00013; gradient 13.09
x: 3.95969; step -0.00013; gradient 13.09
x: 3.95956; step -0.00013; gradient 13.09
x: 3.95943; step -0.00013; gradient 13.09
x: 3.95930; step -0.00013; gradient 13.09
x: 3.95917; step -0.00013; gradient 13.09
x: 3.95904; step -0.00013; gradient 13.09
x: 3.95891; step -0.00013; gradient 13.09
x: 3.95878; step -0.00013; gradient 13.09
x: 3.95865; step -0.00013; gradient 13.09
x: 3.95852; step -0.00013; gradient 13.09
x: 3.95838; step -0.00013; gradient 13.09
x: 3.95825; step -0.00013; gradient 13.09
x: 3.95812; step -0.00013; gradient 13.09
x: 3.95799; step -0.00013; gradient 13.09
x: 3.95786; step -0.00013; gradient 13.09
x: 3.95773; step -0.00013; gradient 13.09
x: 3.95760; step -0.00013; gradient 13.09
x: 3.95747; step -0.00013; gradient 13.09
x: 3.95734; step -0.00013; gradient 13.09
x: 3.95721; step -0.00013; gradient 13.09
x: 3.95708; step -0.00013; gradient 13.09
x: 3.95694; step -0.00013; gradient 13.09
x: 3.95681; step -0.00013; gradient 13.09
x: 3.95668; step -0.00013; gradient 13.09
x: 3.95655; step -0.00013; gradient 13.09
x: 3.95642; step -0.00013; gradient 13.09
x: 3.95629; step -0.00013; gradient 13.09
x: 3.95616; step -0.00013; gradient 13.09
x: 3.95603; step -0.00013; gradient 13.09
x: 3.95590; step -0.00013; gradient 13.09
x: 3.95577; step -0.00013; gradient 13.09
x: 3.95564; step -0.00013; gradient 13.09
x: 3.95551; step -0.00013; gradient 13.09
x: 3.95537; step -0.00013; gradient 13.09
x: 3.95524; step -0.00013; gradient 13.09
x: 3.95511; step -0.00013; gradient 13.09
x: 3.95498; step -0.00013; gradient 13.09
x: 3.95485; step -0.00013; gradient 13.09
x: 3.95472; step -0.00013; gradient 13.09
x: 3.95459; step -0.00013; gradient 13.09
x: 3.95446; step -0.00013; gradient 13.09
x: 3.95433; step -0.00013; gradient 13.08
x: 3.95420; step -0.00013; gradient 13.08
x: 3.95407; step -0.00013; gradient 13.08
x: 3.95393; step -0.00013; gradient 13.08
x: 3.95380; step -0.00013; gradient 13.08
x: 3.95367; step -0.00013; gradient 13.08
x: 3.95354; step -0.00013; gradient 13.08
x: 3.95341; step -0.00013; gradient 13.08
x: 3.95328; step -0.00013; gradient 13.08
x: 3.95315; step -0.00013; gradient 13.08
x: 3.95302; step -0.00013; gradient 13.08
x: 3.95289; step -0.00013; gradient 13.08
x: 3.95276; step -0.00013; gradient 13.08
x: 3.95263; step -0.00013; gradient 13.08
x: 3.95250; step -0.00013; gradient 13.08
x: 3.95236; step -0.00013; gradient 13.08
x: 3.95223; step -0.00013; gradient 13.08
x: 3.95210; step -0.00013; gradient 13.08
x: 3.95197; step -0.00013; gradient 13.08
x: 3.95184; step -0.00013; gradient 13.08
x: 3.95171; step -0.00013; gradient 13.08
x: 3.95158; step -0.00013; gradient 13.08
x: 3.95145; step -0.00013; gradient 13.08
x: 3.95132; step -0.00013; gradient 13.08
x: 3.95119; step -0.00013; gradient 13.08
x: 3.95106; step -0.00013; gradient 13.08
x: 3.95093; step -0.00013; gradient 13.08
x: 3.95080; step -0.00013; gradient 13.08
x: 3.95066; step -0.00013; gradient 13.08
x: 3.95053; step -0.00013; gradient 13.08
x: 3.95040; step -0.00013; gradient 13.08
x: 3.95027; step -0.00013; gradient 13.08
x: 3.95014; step -0.00013; gradient 13.08
x: 3.95001; step -0.00013; gradient 13.08
x: 3.94988; step -0.00013; gradient 13.08
x: 3.94975; step -0.00013; gradient 13.08
x: 3.94962; step -0.00013; gradient 13.08
x: 3.94949; step -0.00013; gradient 13.08
x: 3.94936; step -0.00013; gradient 13.08
x: 3.94923; step -0.00013; gradient 13.08
x: 3.94910; step -0.00013; gradient 13.08
x: 3.94896; step -0.00013; gradient 13.08
x: 3.94883; step -0.00013; gradient 13.07
x: 3.94870; step -0.00013; gradient 13.07
x: 3.94857; step -0.00013; gradient 13.07
x: 3.94844; step -0.00013; gradient 13.07
x: 3.94831; step -0.00013; gradient 13.07
x: 3.94818; step -0.00013; gradient 13.07
x: 3.94805; step -0.00013; gradient 13.07
x: 3.94792; step -0.00013; gradient 13.07
x: 3.94779; step -0.00013; gradient 13.07
x: 3.94766; step -0.00013; gradient 13.07
x: 3.94753; step -0.00013; gradient 13.07
x: 3.94740; step -0.00013; gradient 13.07
x: 3.94726; step -0.00013; gradient 13.07
x: 3.94713; step -0.00013; gradient 13.07
x: 3.94700; step -0.00013; gradient 13.07
x: 3.94687; step -0.00013; gradient 13.07
x: 3.94674; step -0.00013; gradient 13.07
x: 3.94661; step -0.00013; gradient 13.07
x: 3.94648; step -0.00013; gradient 13.07
x: 3.94635; step -0.00013; gradient 13.07
x: 3.94622; step -0.00013; gradient 13.07
x: 3.94609; step -0.00013; gradient 13.07
x: 3.94596; step -0.00013; gradient 13.07
x: 3.94583; step -0.00013; gradient 13.07
x: 3.94570; step -0.00013; gradient 13.07
x: 3.94557; step -0.00013; gradient 13.07
x: 3.94544; step -0.00013; gradient 13.07
x: 3.94530; step -0.00013; gradient 13.07
x: 3.94517; step -0.00013; gradient 13.07
x: 3.94504; step -0.00013; gradient 13.07
x: 3.94491; step -0.00013; gradient 13.07
x: 3.94478; step -0.00013; gradient 13.07
x: 3.94465; step -0.00013; gradient 13.07
x: 3.94452; step -0.00013; gradient 13.07
x: 3.94439; step -0.00013; gradient 13.07
x: 3.94426; step -0.00013; gradient 13.07
x: 3.94413; step -0.00013; gradient 13.07
x: 3.94400; step -0.00013; gradient 13.07
x: 3.94387; step -0.00013; gradient 13.07
x: 3.94374; step -0.00013; gradient 13.07
x: 3.94361; step -0.00013; gradient 13.07
x: 3.94347; step -0.00013; gradient 13.07
x: 3.94334; step -0.00013; gradient 13.06
x: 3.94321; step -0.00013; gradient 13.06
x: 3.94308; step -0.00013; gradient 13.06
x: 3.94295; step -0.00013; gradient 13.06
x: 3.94282; step -0.00013; gradient 13.06
x: 3.94269; step -0.00013; gradient 13.06
x: 3.94256; step -0.00013; gradient 13.06
x: 3.94243; step -0.00013; gradient 13.06
x: 3.94230; step -0.00013; gradient 13.06
x: 3.94217; step -0.00013; gradient 13.06
x: 3.94204; step -0.00013; gradient 13.06
x: 3.94191; step -0.00013; gradient 13.06
x: 3.94178; step -0.00013; gradient 13.06
x: 3.94165; step -0.00013; gradient 13.06
x: 3.94152; step -0.00013; gradient 13.06
x: 3.94138; step -0.00013; gradient 13.06
x: 3.94125; step -0.00013; gradient 13.06
x: 3.94112; step -0.00013; gradient 13.06
x: 3.94099; step -0.00013; gradient 13.06
x: 3.94086; step -0.00013; gradient 13.06
x: 3.94073; step -0.00013; gradient 13.06
x: 3.94060; step -0.00013; gradient 13.06
x: 3.94047; step -0.00013; gradient 13.06
x: 3.94034; step -0.00013; gradient 13.06
x: 3.94021; step -0.00013; gradient 13.06
x: 3.94008; step -0.00013; gradient 13.06
x: 3.93995; step -0.00013; gradient 13.06
x: 3.93982; step -0.00013; gradient 13.06
x: 3.93969; step -0.00013; gradient 13.06
x: 3.93956; step -0.00013; gradient 13.06
x: 3.93943; step -0.00013; gradient 13.06
x: 3.93930; step -0.00013; gradient 13.06
x: 3.93916; step -0.00013; gradient 13.06
x: 3.93903; step -0.00013; gradient 13.06
x: 3.93890; step -0.00013; gradient 13.06
x: 3.93877; step -0.00013; gradient 13.06
x: 3.93864; step -0.00013; gradient 13.06
x: 3.93851; step -0.00013; gradient 13.06
x: 3.93838; step -0.00013; gradient 13.06
x: 3.93825; step -0.00013; gradient 13.06
x: 3.93812; step -0.00013; gradient 13.06
x: 3.93799; step -0.00013; gradient 13.05
x: 3.93786; step -0.00013; gradient 13.05
x: 3.93773; step -0.00013; gradient 13.05
x: 3.93760; step -0.00013; gradient 13.05
x: 3.93747; step -0.00013; gradient 13.05
x: 3.93734; step -0.00013; gradient 13.05
x: 3.93721; step -0.00013; gradient 13.05
x: 3.93708; step -0.00013; gradient 13.05
x: 3.93695; step -0.00013; gradient 13.05
x: 3.93681; step -0.00013; gradient 13.05
x: 3.93668; step -0.00013; gradient 13.05
x: 3.93655; step -0.00013; gradient 13.05
x: 3.93642; step -0.00013; gradient 13.05
x: 3.93629; step -0.00013; gradient 13.05
x: 3.93616; step -0.00013; gradient 13.05
x: 3.93603; step -0.00013; gradient 13.05
x: 3.93590; step -0.00013; gradient 13.05
x: 3.93577; step -0.00013; gradient 13.05
x: 3.93564; step -0.00013; gradient 13.05
x: 3.93551; step -0.00013; gradient 13.05
x: 3.93538; step -0.00013; gradient 13.05
x: 3.93525; step -0.00013; gradient 13.05
x: 3.93512; step -0.00013; gradient 13.05
x: 3.93499; step -0.00013; gradient 13.05
x: 3.93486; step -0.00013; gradient 13.05
x: 3.93473; step -0.00013; gradient 13.05
x: 3.93460; step -0.00013; gradient 13.05
x: 3.93447; step -0.00013; gradient 13.05
x: 3.93434; step -0.00013; gradient 13.05
x: 3.93420; step -0.00013; gradient 13.05
x: 3.93407; step -0.00013; gradient 13.05
x: 3.93394; step -0.00013; gradient 13.05
x: 3.93381; step -0.00013; gradient 13.05
x: 3.93368; step -0.00013; gradient 13.05
x: 3.93355; step -0.00013; gradient 13.05
x: 3.93342; step -0.00013; gradient 13.05
x: 3.93329; step -0.00013; gradient 13.05
x: 3.93316; step -0.00013; gradient 13.05
x: 3.93303; step -0.00013; gradient 13.05
x: 3.93290; step -0.00013; gradient 13.05
x: 3.93277; step -0.00013; gradient 13.04
x: 3.93264; step -0.00013; gradient 13.04
x: 3.93251; step -0.00013; gradient 13.04
x: 3.93238; step -0.00013; gradient 13.04
x: 3.93225; step -0.00013; gradient 13.04
x: 3.93212; step -0.00013; gradient 13.04
x: 3.93199; step -0.00013; gradient 13.04
x: 3.93186; step -0.00013; gradient 13.04
x: 3.93173; step -0.00013; gradient 13.04
x: 3.93160; step -0.00013; gradient 13.04
x: 3.93147; step -0.00013; gradient 13.04
x: 3.93133; step -0.00013; gradient 13.04
x: 3.93120; step -0.00013; gradient 13.04
x: 3.93107; step -0.00013; gradient 13.04
x: 3.93094; step -0.00013; gradient 13.04
x: 3.93081; step -0.00013; gradient 13.04
x: 3.93068; step -0.00013; gradient 13.04
x: 3.93055; step -0.00013; gradient 13.04
x: 3.93042; step -0.00013; gradient 13.04
x: 3.93029; step -0.00013; gradient 13.04
x: 3.93016; step -0.00013; gradient 13.04
x: 3.93003; step -0.00013; gradient 13.04
x: 3.92990; step -0.00013; gradient 13.04
x: 3.92977; step -0.00013; gradient 13.04
x: 3.92964; step -0.00013; gradient 13.04
x: 3.92951; step -0.00013; gradient 13.04
x: 3.92938; step -0.00013; gradient 13.04
x: 3.92925; step -0.00013; gradient 13.04
x: 3.92912; step -0.00013; gradient 13.04
x: 3.92899; step -0.00013; gradient 13.04
x: 3.92886; step -0.00013; gradient 13.04
x: 3.92873; step -0.00013; gradient 13.04
x: 3.92860; step -0.00013; gradient 13.04
x: 3.92847; step -0.00013; gradient 13.04
x: 3.92834; step -0.00013; gradient 13.04
x: 3.92821; step -0.00013; gradient 13.04
x: 3.92808; step -0.00013; gradient 13.04
x: 3.92794; step -0.00013; gradient 13.04
x: 3.92781; step -0.00013; gradient 13.04
x: 3.92768; step -0.00013; gradient 13.04
x: 3.92755; step -0.00013; gradient 13.03
x: 3.92742; step -0.00013; gradient 13.03
x: 3.92729; step -0.00013; gradient 13.03
x: 3.92716; step -0.00013; gradient 13.03
x: 3.92703; step -0.00013; gradient 13.03
x: 3.92690; step -0.00013; gradient 13.03
x: 3.92677; step -0.00013; gradient 13.03
x: 3.92664; step -0.00013; gradient 13.03
x: 3.92651; step -0.00013; gradient 13.03
x: 3.92638; step -0.00013; gradient 13.03
x: 3.92625; step -0.00013; gradient 13.03
x: 3.92612; step -0.00013; gradient 13.03
x: 3.92599; step -0.00013; gradient 13.03
x: 3.92586; step -0.00013; gradient 13.03
x: 3.92573; step -0.00013; gradient 13.03
x: 3.92560; step -0.00013; gradient 13.03
x: 3.92547; step -0.00013; gradient 13.03
x: 3.92534; step -0.00013; gradient 13.03
x: 3.92521; step -0.00013; gradient 13.03
x: 3.92508; step -0.00013; gradient 13.03
x: 3.92495; step -0.00013; gradient 13.03
x: 3.92482; step -0.00013; gradient 13.03
x: 3.92469; step -0.00013; gradient 13.03
x: 3.92456; step -0.00013; gradient 13.03
x: 3.92443; step -0.00013; gradient 13.03
x: 3.92430; step -0.00013; gradient 13.03
x: 3.92417; step -0.00013; gradient 13.03
x: 3.92404; step -0.00013; gradient 13.03
x: 3.92390; step -0.00013; gradient 13.03
x: 3.92377; step -0.00013; gradient 13.03
x: 3.92364; step -0.00013; gradient 13.03
x: 3.92351; step -0.00013; gradient 13.03
x: 3.92338; step -0.00013; gradient 13.03
x: 3.92325; step -0.00013; gradient 13.03
x: 3.92312; step -0.00013; gradient 13.03
x: 3.92299; step -0.00013; gradient 13.03
x: 3.92286; step -0.00013; gradient 13.03
x: 3.92273; step -0.00013; gradient 13.03
x: 3.92260; step -0.00013; gradient 13.03
x: 3.92247; step -0.00013; gradient 13.02
x: 3.92234; step -0.00013; gradient 13.02
x: 3.92221; step -0.00013; gradient 13.02
x: 3.92208; step -0.00013; gradient 13.02
x: 3.92195; step -0.00013; gradient 13.02
x: 3.92182; step -0.00013; gradient 13.02
x: 3.92169; step -0.00013; gradient 13.02
x: 3.92156; step -0.00013; gradient 13.02
x: 3.92143; step -0.00013; gradient 13.02
x: 3.92130; step -0.00013; gradient 13.02
x: 3.92117; step -0.00013; gradient 13.02
x: 3.92104; step -0.00013; gradient 13.02
x: 3.92091; step -0.00013; gradient 13.02
x: 3.92078; step -0.00013; gradient 13.02
x: 3.92065; step -0.00013; gradient 13.02
x: 3.92052; step -0.00013; gradient 13.02
x: 3.92039; step -0.00013; gradient 13.02
x: 3.92026; step -0.00013; gradient 13.02
x: 3.92013; step -0.00013; gradient 13.02
x: 3.92000; step -0.00013; gradient 13.02
x: 3.91987; step -0.00013; gradient 13.02
x: 3.91974; step -0.00013; gradient 13.02
x: 3.91961; step -0.00013; gradient 13.02
x: 3.91948; step -0.00013; gradient 13.02
x: 3.91935; step -0.00013; gradient 13.02
x: 3.91922; step -0.00013; gradient 13.02
x: 3.91909; step -0.00013; gradient 13.02
x: 3.91896; step -0.00013; gradient 13.02
x: 3.91883; step -0.00013; gradient 13.02
x: 3.91870; step -0.00013; gradient 13.02
x: 3.91857; step -0.00013; gradient 13.02
x: 3.91844; step -0.00013; gradient 13.02
x: 3.91831; step -0.00013; gradient 13.02
x: 3.91818; step -0.00013; gradient 13.02
x: 3.91805; step -0.00013; gradient 13.02
x: 3.91791; step -0.00013; gradient 13.02
x: 3.91778; step -0.00013; gradient 13.02
x: 3.91765; step -0.00013; gradient 13.02
x: 3.91752; step -0.00013; gradient 13.02
x: 3.91739; step -0.00013; gradient 13.01
x: 3.91726; step -0.00013; gradient 13.01
x: 3.91713; step -0.00013; gradient 13.01
x: 3.91700; step -0.00013; gradient 13.01
x: 3.91687; step -0.00013; gradient 13.01
x: 3.91674; step -0.00013; gradient 13.01
x: 3.91661; step -0.00013; gradient 13.01
x: 3.91648; step -0.00013; gradient 13.01
x: 3.91635; step -0.00013; gradient 13.01
x: 3.91622; step -0.00013; gradient 13.01
x: 3.91609; step -0.00013; gradient 13.01
x: 3.91596; step -0.00013; gradient 13.01
x: 3.91583; step -0.00013; gradient 13.01
x: 3.91570; step -0.00013; gradient 13.01
x: 3.91557; step -0.00013; gradient 13.01
x: 3.91544; step -0.00013; gradient 13.01
x: 3.91531; step -0.00013; gradient 13.01
x: 3.91518; step -0.00013; gradient 13.01
x: 3.91505; step -0.00013; gradient 13.01
x: 3.91492; step -0.00013; gradient 13.01
x: 3.91479; step -0.00013; gradient 13.01
x: 3.91466; step -0.00013; gradient 13.01
x: 3.91453; step -0.00013; gradient 13.01
x: 3.91440; step -0.00013; gradient 13.01
x: 3.91427; step -0.00013; gradient 13.01
x: 3.91414; step -0.00013; gradient 13.01
x: 3.91401; step -0.00013; gradient 13.01
x: 3.91388; step -0.00013; gradient 13.01
x: 3.91375; step -0.00013; gradient 13.01
x: 3.91362; step -0.00013; gradient 13.01
x: 3.91349; step -0.00013; gradient 13.01
x: 3.91336; step -0.00013; gradient 13.01
x: 3.91323; step -0.00013; gradient 13.01
x: 3.91310; step -0.00013; gradient 13.01
x: 3.91297; step -0.00013; gradient 13.01
x: 3.91284; step -0.00013; gradient 13.01
x: 3.91271; step -0.00013; gradient 13.01
x: 3.91258; step -0.00013; gradient 13.01
x: 3.91245; step -0.00013; gradient 13.00
x: 3.91232; step -0.00013; gradient 13.00
x: 3.91219; step -0.00013; gradient 13.00
x: 3.91206; step -0.00013; gradient 13.00
x: 3.91193; step -0.00013; gradient 13.00
x: 3.91180; step -0.00013; gradient 13.00
x: 3.91167; step -0.00013; gradient 13.00
x: 3.91154; step -0.00013; gradient 13.00
x: 3.91141; step -0.00013; gradient 13.00
x: 3.91128; step -0.00013; gradient 13.00
x: 3.91115; step -0.00013; gradient 13.00
x: 3.91102; step -0.00013; gradient 13.00
x: 3.91089; step -0.00013; gradient 13.00
x: 3.91076; step -0.00013; gradient 13.00
x: 3.91063; step -0.00013; gradient 13.00
x: 3.91050; step -0.00013; gradient 13.00
x: 3.91037; step -0.00013; gradient 13.00
x: 3.91024; step -0.00013; gradient 13.00
x: 3.91011; step -0.00013; gradient 13.00
x: 3.90998; step -0.00013; gradient 13.00
x: 3.90985; step -0.00013; gradient 13.00
x: 3.90972; step -0.00013; gradient 13.00
x: 3.90959; step -0.00013; gradient 13.00
x: 3.90946; step -0.00013; gradient 13.00
x: 3.90933; step -0.00013; gradient 13.00
x: 3.90920; step -0.00013; gradient 13.00
x: 3.90907; step -0.00013; gradient 13.00
x: 3.90894; step -0.00013; gradient 13.00
x: 3.90881; step -0.00013; gradient 13.00
x: 3.90868; step -0.00013; gradient 13.00
x: 3.90855; step -0.00013; gradient 13.00
x: 3.90842; step -0.00013; gradient 13.00
x: 3.90829; step -0.00013; gradient 13.00
x: 3.90816; step -0.00013; gradient 13.00
x: 3.90803; step -0.00013; gradient 13.00
x: 3.90790; step -0.00013; gradient 13.00
x: 3.90777; step -0.00013; gradient 13.00
x: 3.90764; step -0.00013; gradient 12.99
x: 3.90751; step -0.00013; gradient 12.99
x: 3.90738; step -0.00013; gradient 12.99
x: 3.90725; step -0.00013; gradient 12.99
x: 3.90712; step -0.00013; gradient 12.99
x: 3.90699; step -0.00013; gradient 12.99
x: 3.90686; step -0.00013; gradient 12.99
x: 3.90673; step -0.00013; gradient 12.99
x: 3.90660; step -0.00013; gradient 12.99
x: 3.90647; step -0.00013; gradient 12.99
x: 3.90634; step -0.00013; gradient 12.99
x: 3.90621; step -0.00013; gradient 12.99
x: 3.90608; step -0.00013; gradient 12.99
x: 3.90595; step -0.00013; gradient 12.99
x: 3.90582; step -0.00013; gradient 12.99
x: 3.90569; step -0.00013; gradient 12.99
x: 3.90556; step -0.00013; gradient 12.99
x: 3.90543; step -0.00013; gradient 12.99
x: 3.90530; step -0.00013; gradient 12.99
x: 3.90517; step -0.00013; gradient 12.99
x: 3.90504; step -0.00013; gradient 12.99
x: 3.90491; step -0.00013; gradient 12.99
x: 3.90478; step -0.00013; gradient 12.99
x: 3.90465; step -0.00013; gradient 12.99
x: 3.90452; step -0.00013; gradient 12.99
x: 3.90439; step -0.00013; gradient 12.99
x: 3.90426; step -0.00013; gradient 12.99
x: 3.90413; step -0.00013; gradient 12.99
x: 3.90400; step -0.00013; gradient 12.99
x: 3.90387; step -0.00013; gradient 12.99
x: 3.90374; step -0.00013; gradient 12.99
x: 3.90361; step -0.00013; gradient 12.99
x: 3.90348; step -0.00013; gradient 12.99
x: 3.90335; step -0.00013; gradient 12.99
x: 3.90322; step -0.00013; gradient 12.99
x: 3.90309; step -0.00013; gradient 12.99
x: 3.90296; step -0.00013; gradient 12.99
x: 3.90283; step -0.00013; gradient 12.98
x: 3.90270; step -0.00013; gradient 12.98
x: 3.90257; step -0.00013; gradient 12.98
x: 3.90244; step -0.00013; gradient 12.98
x: 3.90231; step -0.00013; gradient 12.98
x: 3.90218; step -0.00013; gradient 12.98
x: 3.90206; step -0.00013; gradient 12.98
x: 3.90193; step -0.00013; gradient 12.98
x: 3.90180; step -0.00013; gradient 12.98
x: 3.90167; step -0.00013; gradient 12.98
x: 3.90154; step -0.00013; gradient 12.98
x: 3.90141; step -0.00013; gradient 12.98
x: 3.90128; step -0.00013; gradient 12.98
x: 3.90115; step -0.00013; gradient 12.98
x: 3.90102; step -0.00013; gradient 12.98
x: 3.90089; step -0.00013; gradient 12.98
x: 3.90076; step -0.00013; gradient 12.98
x: 3.90063; step -0.00013; gradient 12.98
x: 3.90050; step -0.00013; gradient 12.98
x: 3.90037; step -0.00013; gradient 12.98
x: 3.90024; step -0.00013; gradient 12.98
x: 3.90011; step -0.00013; gradient 12.98
x: 3.89998; step -0.00013; gradient 12.98
x: 3.89985; step -0.00013; gradient 12.98
x: 3.89972; step -0.00013; gradient 12.98
x: 3.89959; step -0.00013; gradient 12.98
x: 3.89946; step -0.00013; gradient 12.98
x: 3.89933; step -0.00013; gradient 12.98
x: 3.89920; step -0.00013; gradient 12.98
x: 3.89907; step -0.00013; gradient 12.98
x: 3.89894; step -0.00013; gradient 12.98
x: 3.89881; step -0.00013; gradient 12.98
x: 3.89868; step -0.00013; gradient 12.98
x: 3.89855; step -0.00013; gradient 12.98
x: 3.89842; step -0.00013; gradient 12.98
x: 3.89829; step -0.00013; gradient 12.98
x: 3.89816; step -0.00013; gradient 12.98
x: 3.89803; step -0.00013; gradient 12.97
x: 3.89790; step -0.00013; gradient 12.97
x: 3.89777; step -0.00013; gradient 12.97
x: 3.89764; step -0.00013; gradient 12.97
x: 3.89751; step -0.00013; gradient 12.97
x: 3.89738; step -0.00013; gradient 12.97
x: 3.89725; step -0.00013; gradient 12.97
x: 3.89712; step -0.00013; gradient 12.97
x: 3.89699; step -0.00013; gradient 12.97
x: 3.89686; step -0.00013; gradient 12.97
x: 3.89673; step -0.00013; gradient 12.97
x: 3.89660; step -0.00013; gradient 12.97
x: 3.89647; step -0.00013; gradient 12.97
x: 3.89635; step -0.00013; gradient 12.97
x: 3.89622; step -0.00013; gradient 12.97
x: 3.89609; step -0.00013; gradient 12.97
x: 3.89596; step -0.00013; gradient 12.97
x: 3.89583; step -0.00013; gradient 12.97
x: 3.89570; step -0.00013; gradient 12.97
x: 3.89557; step -0.00013; gradient 12.97
x: 3.89544; step -0.00013; gradient 12.97
x: 3.89531; step -0.00013; gradient 12.97
x: 3.89518; step -0.00013; gradient 12.97
x: 3.89505; step -0.00013; gradient 12.97
x: 3.89492; step -0.00013; gradient 12.97
x: 3.89479; step -0.00013; gradient 12.97
x: 3.89466; step -0.00013; gradient 12.97
x: 3.89453; step -0.00013; gradient 12.97
x: 3.89440; step -0.00013; gradient 12.97
x: 3.89427; step -0.00013; gradient 12.97
x: 3.89414; step -0.00013; gradient 12.97
x: 3.89401; step -0.00013; gradient 12.97
x: 3.89388; step -0.00013; gradient 12.97
x: 3.89375; step -0.00013; gradient 12.97
x: 3.89362; step -0.00013; gradient 12.97
x: 3.89349; step -0.00013; gradient 12.97
x: 3.89336; step -0.00013; gradient 12.96
x: 3.89323; step -0.00013; gradient 12.96
x: 3.89310; step -0.00013; gradient 12.96
x: 3.89297; step -0.00013; gradient 12.96
x: 3.89284; step -0.00013; gradient 12.96
x: 3.89271; step -0.00013; gradient 12.96
x: 3.89258; step -0.00013; gradient 12.96
x: 3.89245; step -0.00013; gradient 12.96
x: 3.89233; step -0.00013; gradient 12.96
x: 3.89220; step -0.00013; gradient 12.96
x: 3.89207; step -0.00013; gradient 12.96
x: 3.89194; step -0.00013; gradient 12.96
x: 3.89181; step -0.00013; gradient 12.96
x: 3.89168; step -0.00013; gradient 12.96
x: 3.89155; step -0.00013; gradient 12.96
x: 3.89142; step -0.00013; gradient 12.96
x: 3.89129; step -0.00013; gradient 12.96
x: 3.89116; step -0.00013; gradient 12.96
x: 3.89103; step -0.00013; gradient 12.96
x: 3.89090; step -0.00013; gradient 12.96
x: 3.89077; step -0.00013; gradient 12.96
x: 3.89064; step -0.00013; gradient 12.96
x: 3.89051; step -0.00013; gradient 12.96
x: 3.89038; step -0.00013; gradient 12.96
x: 3.89025; step -0.00013; gradient 12.96
x: 3.89012; step -0.00013; gradient 12.96
x: 3.88999; step -0.00013; gradient 12.96
x: 3.88986; step -0.00013; gradient 12.96
x: 3.88973; step -0.00013; gradient 12.96
x: 3.88960; step -0.00013; gradient 12.96
x: 3.88947; step -0.00013; gradient 12.96
x: 3.88934; step -0.00013; gradient 12.96
x: 3.88922; step -0.00013; gradient 12.96
x: 3.88909; step -0.00013; gradient 12.96
x: 3.88896; step -0.00013; gradient 12.96
x: 3.88883; step -0.00013; gradient 12.95
x: 3.88870; step -0.00013; gradient 12.95
x: 3.88857; step -0.00013; gradient 12.95
x: 3.88844; step -0.00013; gradient 12.95
x: 3.88831; step -0.00013; gradient 12.95
x: 3.88818; step -0.00013; gradient 12.95
x: 3.88805; step -0.00013; gradient 12.95
x: 3.88792; step -0.00013; gradient 12.95
x: 3.88779; step -0.00013; gradient 12.95
x: 3.88766; step -0.00013; gradient 12.95
x: 3.88753; step -0.00013; gradient 12.95
x: 3.88740; step -0.00013; gradient 12.95
x: 3.88727; step -0.00013; gradient 12.95
x: 3.88714; step -0.00013; gradient 12.95
x: 3.88701; step -0.00013; gradient 12.95
x: 3.88688; step -0.00013; gradient 12.95
x: 3.88675; step -0.00013; gradient 12.95
x: 3.88662; step -0.00013; gradient 12.95
x: 3.88649; step -0.00013; gradient 12.95
x: 3.88637; step -0.00013; gradient 12.95
x: 3.88624; step -0.00013; gradient 12.95
x: 3.88611; step -0.00013; gradient 12.95
x: 3.88598; step -0.00013; gradient 12.95
x: 3.88585; step -0.00013; gradient 12.95
x: 3.88572; step -0.00013; gradient 12.95
x: 3.88559; step -0.00013; gradient 12.95
x: 3.88546; step -0.00013; gradient 12.95
x: 3.88533; step -0.00013; gradient 12.95
x: 3.88520; step -0.00013; gradient 12.95
x: 3.88507; step -0.00013; gradient 12.95
x: 3.88494; step -0.00013; gradient 12.95
x: 3.88481; step -0.00013; gradient 12.95
x: 3.88468; step -0.00013; gradient 12.95
x: 3.88455; step -0.00013; gradient 12.95
x: 3.88442; step -0.00013; gradient 12.95
x: 3.88429; step -0.00013; gradient 12.94
x: 3.88416; step -0.00013; gradient 12.94
x: 3.88403; step -0.00013; gradient 12.94
x: 3.88391; step -0.00013; gradient 12.94
x: 3.88378; step -0.00013; gradient 12.94
x: 3.88365; step -0.00013; gradient 12.94
x: 3.88352; step -0.00013; gradient 12.94
x: 3.88339; step -0.00013; gradient 12.94
x: 3.88326; step -0.00013; gradient 12.94
x: 3.88313; step -0.00013; gradient 12.94
x: 3.88300; step -0.00013; gradient 12.94
x: 3.88287; step -0.00013; gradient 12.94
x: 3.88274; step -0.00013; gradient 12.94
x: 3.88261; step -0.00013; gradient 12.94
x: 3.88248; step -0.00013; gradient 12.94
x: 3.88235; step -0.00013; gradient 12.94
x: 3.88222; step -0.00013; gradient 12.94
x: 3.88209; step -0.00013; gradient 12.94
x: 3.88196; step -0.00013; gradient 12.94
x: 3.88183; step -0.00013; gradient 12.94
x: 3.88171; step -0.00013; gradient 12.94
x: 3.88158; step -0.00013; gradient 12.94
x: 3.88145; step -0.00013; gradient 12.94
x: 3.88132; step -0.00013; gradient 12.94
x: 3.88119; step -0.00013; gradient 12.94
x: 3.88106; step -0.00013; gradient 12.94
x: 3.88093; step -0.00013; gradient 12.94
x: 3.88080; step -0.00013; gradient 12.94
x: 3.88067; step -0.00013; gradient 12.94
x: 3.88054; step -0.00013; gradient 12.94
x: 3.88041; step -0.00013; gradient 12.94
x: 3.88028; step -0.00013; gradient 12.94
x: 3.88015; step -0.00013; gradient 12.94
x: 3.88002; step -0.00013; gradient 12.94
x: 3.87989; step -0.00013; gradient 12.94
x: 3.87976; step -0.00013; gradient 12.93
x: 3.87964; step -0.00013; gradient 12.93
x: 3.87951; step -0.00013; gradient 12.93
x: 3.87938; step -0.00013; gradient 12.93
x: 3.87925; step -0.00013; gradient 12.93
x: 3.87912; step -0.00013; gradient 12.93
x: 3.87899; step -0.00013; gradient 12.93
x: 3.87886; step -0.00013; gradient 12.93
x: 3.87873; step -0.00013; gradient 12.93
x: 3.87860; step -0.00013; gradient 12.93
x: 3.87847; step -0.00013; gradient 12.93
x: 3.87834; step -0.00013; gradient 12.93
x: 3.87821; step -0.00013; gradient 12.93
x: 3.87808; step -0.00013; gradient 12.93
x: 3.87795; step -0.00013; gradient 12.93
x: 3.87782; step -0.00013; gradient 12.93
x: 3.87770; step -0.00013; gradient 12.93
x: 3.87757; step -0.00013; gradient 12.93
x: 3.87744; step -0.00013; gradient 12.93
x: 3.87731; step -0.00013; gradient 12.93
x: 3.87718; step -0.00013; gradient 12.93
x: 3.87705; step -0.00013; gradient 12.93
x: 3.87692; step -0.00013; gradient 12.93
x: 3.87679; step -0.00013; gradient 12.93
x: 3.87666; step -0.00013; gradient 12.93
x: 3.87653; step -0.00013; gradient 12.93
x: 3.87640; step -0.00013; gradient 12.93
x: 3.87627; step -0.00013; gradient 12.93
x: 3.87614; step -0.00013; gradient 12.93
x: 3.87601; step -0.00013; gradient 12.93
x: 3.87589; step -0.00013; gradient 12.93
x: 3.87576; step -0.00013; gradient 12.93
x: 3.87563; step -0.00013; gradient 12.93
x: 3.87550; step -0.00013; gradient 12.93
x: 3.87537; step -0.00013; gradient 12.92
x: 3.87524; step -0.00013; gradient 12.92
x: 3.87511; step -0.00013; gradient 12.92
x: 3.87498; step -0.00013; gradient 12.92
x: 3.87485; step -0.00013; gradient 12.92
x: 3.87472; step -0.00013; gradient 12.92
x: 3.87459; step -0.00013; gradient 12.92
x: 3.87446; step -0.00013; gradient 12.92
x: 3.87433; step -0.00013; gradient 12.92
x: 3.87421; step -0.00013; gradient 12.92
x: 3.87408; step -0.00013; gradient 12.92
x: 3.87395; step -0.00013; gradient 12.92
x: 3.87382; step -0.00013; gradient 12.92
x: 3.87369; step -0.00013; gradient 12.92
x: 3.87356; step -0.00013; gradient 12.92
x: 3.87343; step -0.00013; gradient 12.92
x: 3.87330; step -0.00013; gradient 12.92
x: 3.87317; step -0.00013; gradient 12.92
x: 3.87304; step -0.00013; gradient 12.92
x: 3.87291; step -0.00013; gradient 12.92
x: 3.87278; step -0.00013; gradient 12.92
x: 3.87265; step -0.00013; gradient 12.92
x: 3.87253; step -0.00013; gradient 12.92
x: 3.87240; step -0.00013; gradient 12.92
x: 3.87227; step -0.00013; gradient 12.92
x: 3.87214; step -0.00013; gradient 12.92
x: 3.87201; step -0.00013; gradient 12.92
x: 3.87188; step -0.00013; gradient 12.92
x: 3.87175; step -0.00013; gradient 12.92
x: 3.87162; step -0.00013; gradient 12.92
x: 3.87149; step -0.00013; gradient 12.92
x: 3.87136; step -0.00013; gradient 12.92
x: 3.87123; step -0.00013; gradient 12.92
x: 3.87110; step -0.00013; gradient 12.92
x: 3.87098; step -0.00013; gradient 12.91
x: 3.87085; step -0.00013; gradient 12.91
x: 3.87072; step -0.00013; gradient 12.91
x: 3.87059; step -0.00013; gradient 12.91
x: 3.87046; step -0.00013; gradient 12.91
x: 3.87033; step -0.00013; gradient 12.91
x: 3.87020; step -0.00013; gradient 12.91
x: 3.87007; step -0.00013; gradient 12.91
x: 3.86994; step -0.00013; gradient 12.91
x: 3.86981; step -0.00013; gradient 12.91
x: 3.86968; step -0.00013; gradient 12.91
Minimum at 3.8695553023659817
As you can see, by doing this, we save ourselves from trying a very large number of other potential x values, and we focus in on the minimum very quickly.
This is just one method among many for optimizing our search for the minimum of
a function. Now you know what kind of thing it is doing, we will just let
minimize do its job for us:
# Use float to show us the final result in higher precision
result = minimize(calc_rmse, 100)
float(result.x)
/tmp/ipykernel_5511/3243124131.py:3: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
float(result.x)
3.0474986374788746