---
jupyter:
  jupytext:
    notebook_metadata_filter: all,-language_info
    split_at_heading: true
    text_representation:
      extension: .Rmd
      format_name: rmarkdown
      format_version: '1.2'
      jupytext_version: 1.14.1
  kernelspec:
    display_name: Python 3 (ipykernel)
    language: python
    name: python3
---

# Using minimize

We have already used the `minimize` function a few times:

```{python}
from scipy.optimize import minimize
```

Now we pause to look at how it works, and how best to use it.

Let's go back to the problem from [finding lines](finding_lines):

```{python}
# Our usual imports and configuration.
import numpy as np
import pandas as pd
pd.set_option('mode.copy_on_write', True)
import matplotlib.pyplot as plt
# Make plots look a little bit more fancy
plt.style.use('fivethirtyeight')
```

We used the [students ratings dataset](../data/rate_my_professors) dataset.
Download the data file via
{download}`rate_my_course.csv <../data/rate_my_course.csv>`.

We were looking for the best slope to relate the Easiness ratings to the
Overall Quality ratings.

```{python}
# Read the data file, get columns as arrays.
ratings = pd.read_csv('rate_my_course.csv')
easiness = np.array(ratings['Easiness'])
quality = np.array(ratings['Overall Quality'])
plt.plot(easiness, quality, 'o')
plt.xlabel('Easiness')
plt.ylabel('Overall quality')
```

Here is the function we used to calculate root means squared error (RMSE),
adapted for `minimize`.

```{python}
def calc_rmse_for_minimize(c_s):
    # c_s has two elements, the intercept c and the slope s.
    c = c_s[0]
    s = c_s[1]
    predicted_quality = c + easiness * s
    errors = quality - predicted_quality
    return np.sqrt(np.mean(errors ** 2))
```

We called `minimize` to find the best intercept and slope:

```{python}
min_res = minimize(calc_rmse_for_minimize, [2.25, 0.47])
min_res
```

## The result of `minimize`

Notice the thing that `minimize` returns is a special kind of thing that stores
information about the result:

```{python}
# The value that comes back is a special thing to contain minimize results:
type(min_res)
```

Among the interesting thing that the results contain is the attribute `fun`.
This value is the final minimal value of the function we are trying to minimize
— here the RMSE given an intercept and slope:

```{python}
# The minimal value that `minimize` could find for our function.
min_res.fun
```

We also have an array with the values for the intercept and slope that give the minimal value:

```{python}
min_res.x
```

We confirm that `min_res.fun` is indeed the value we get from our function given the intercept, slope array in `min_res.x`:

```{python}
calc_rmse_for_minimize(min_res.x)
```

## The function to minimize

`calc_rmse_for_minimize` is a *function*:

```{python}
type(calc_rmse_for_minimize)
```

We pass the function to `minimize` as an argument for it to use.

We can call the function we pass to `minimize` — the *objective function* — in
the sense that our objective is to minimize this function.

How is `minimize` using our objective function?   Clearly `minimize` is
[calling](../code-basics/Calls) the function, but what arguments is it
sending?  How many times does it call the function?

To find out, we can stick something inside the objective function to tell us
each time it is called:

```{python}
def rmse_func_for_info(c_s):
    # Print the argument that minimize sent.
    print('Called with', c_s, 'of type', type(c_s))
    # The rest of the function is the same as the original above.
    # c_s has two elements, the intercept c and the slope s.
    c = c_s[0]
    s = c_s[1]
    predicted_quality = c + easiness * s
    errors = quality - predicted_quality
    return np.sqrt(np.mean(errors ** 2))
```

The we call `minimize` with the new objective function:

```{python}
min_res = minimize(rmse_func_for_info, [2.25, 0.47])
min_res
```

This shows that:

* `minimize` calls our function multiple times, as it searches for the values
  of intercept and slope giving the minimum RMSE.
* At each call, it passes a single argument that is an *array* containing the
  two values (intercept and slope).

Looking carefully, we see signs that `minimize` is trying small changes in the
slope or intercept, presumably to calculate the gradient — as we saw in the
[optimization](optimization) page — but that is not our concern here.

The values in the array that `minimize` passes are the *parameters* that
minimize is trying to optimize — in our case, the intercept and slope.  Call
this the *parameter array*.

## Tricks for using minimize

(unpacking)=

### Unpacking

Near the top of our function, we have these two lines of code:

```
    c = c_s[0]
    s = c_s[1]
```

This is where we take set the intercept from the first value of the *parameter
array*, and the slope from the second.

It turns out there is neat and versatile way of doing this in Python, called
*unpacking*.   Consider this array:

```{python}
c_s = np.array([2, 1])
```

We can *unpack* these two values into variables like this:

```{python}
# Unpacking!
c, s = c_s
```

The right hand side contains two values (in an array).  The left hand side has
two variable names, separated by a comma.  Python take the two values from the right hand side, and puts them into the variables on the left:

```{python}
print('c is', c)
print('s_is', s)
```

The thing on the right hand side can be anything that is a sequence of two things.  For example, it can also be a list:

```{python}
my_list = [6, 7]
my_var1, my_var2 = my_list
print('my_var1 is', my_var1)
print('my_var2 is', my_var2)
```

There can be three or four or any other number of variables on the left, as long as there is a matching number of elements in the thing on the right:

```{python}
list2 = [10, 100, 10000]
w, x, y = list2
print('w is', w)
print('x is', x)
print('y is', y)
```

The number of elements must match the number of variables:

```{python tags=c("raises-exception")}
# Error - three variables on the left, two elements on the right.
p, q, r = [1, 2]
```

### Using unpacking in minimize

Unpacking has two good uses with `minimize`.  The first is that we can make our function to minimize a little neater:

```{python}
def rmse_with_unpacking(c_s):
    # c_s has two elements, the intercept c and the slope s.
    # Use unpacking!
    c, s = c_s
    predicted_quality = c + easiness * s
    errors = quality - predicted_quality
    return np.sqrt(np.mean(errors ** 2))
```

The second is, we can use unpacking to — er — unpack the best fit intercept and
slope from the result of `minimize`.  Remember, `minimize` returns a result
value, that includes an array `x` with the parameters minimizing our function:

```{python}
m_r = minimize(rmse_with_unpacking, [2.25, 0.47])
m_r
```

We can get the best-fit intercept and slope values by unpacking:

```{python}
best_c, best_s = m_r.x
print(best_c)
print(best_s)
```

### Other arguments to our function


At the moment, using `minimize` is a little inconvenient, because we have to make a separate cost (objective) function for each set of x and y values.

Here is our new, neater cost function to find the best slope and intercept for `easiness` and `quality`:

```{python}
def cost_easy_quality(c_s):
    # c_s has two elements, the intercept c and the slope s.
    c, s = c_s
    predicted_quality = c + easiness * s
    errors = quality - predicted_quality
    return np.sqrt(np.mean(errors ** 2))
```

We find the intercept and slope — and get the usual values.

```{python}
# Show the best intercept and slope for easiness and quality.
minimize(cost_easy_quality, [2.25, 0.47]).x
```

Now let's imagine we are interested in the relationship of `easiness` and `clarity`:

```{python}
clarity = np.array(ratings['Overall Quality'])
clarity[:10]
```

Notice that our `cost_easy_quality` function uses the top-level notebook
variables `easiness` and `quality` within the function.  It can only work on
these variables, and no others.  So, if we want to get the slope and intercept
for `easiness` and `clarity`, instead of `easiness` and `quality`, we have to
write another almost-identical function, like this:

```{python}
def cost_easy_clarity(c_s):
    # c_s has two elements, the intercept c and the slope s.
    c, s = c_s
    predicted_clarity = c + easiness * s
    errors = clarity - predicted_clarity
    return np.sqrt(np.mean(errors ** 2))
```

```{python}
# Show the best intercept and slope for easiness and clarity.
minimize(cost_easy_clarity, [2.25, 0.47]).x
```

How can we avoid writing such near-identical functions for each set of x and y values for which we want a best-fit line?


## Generalizing the cost-function with the args argument

Inspect the help for `minimize` with `minimize?`.   You will notice that there
is an argument we can pass to `minimize` called `args`.  These are "Extra
arguments passed to the objective function".  In fact, this is very useful to
make our objective function more general.  First we try this silly use of
`args` where we will pass a couple of useless bits of text to our objective
function:

```{python}
def rmse_with_extra_args(c_s, v1, v2):
    # An objective function with some useless extra arguments.
    print('c_s is', c_s, '; v1 is', v1, '; v2 is', v2)
    c, s = c_s
    predicted_quality = c + easiness * s
    errors = quality - predicted_quality
    return np.sqrt(np.mean(errors ** 2))
```

Now we tell `minimize` to pass some value for `v1` and `v2` for every call to
the objective function. We do that by passing `v1` and `v2` inside a *tuple*. A
*tuple* is a data type in Python that is much like a list, in that it contains
sequences of values, we create tuples using parentheses `()` rather than square
brackets.

Here is our argument tuple:

```{python}
v1 = 'Doctor'
v2 = 'Strange'
# Make the argument tuple. Notice the parentheses.
extras = (v1, v2)
extras
```

Here we pass the argument tuple to `minimize`, to ask `minimize` to pass these
extra arguments to the objective function, each time it calls the function:


```{python}
# args has two values, one that becomes "v1" and the other that becomes "v2".
minimize(rmse_with_extra_args, [2.25, 0.47], args=extras)
```

To be more compact, we can assemble and pass the `args` argument in one go:

```{python}
# Compiling and passing args in one line.
minimize(rmse_with_extra_args, [2.25, 0.47], args=('Doctor', 'Strange'))
```

Notice that, each time `minimize` calls `rmse_with_extra_args` it passes the
first value of `args` as the second argument to the objective function (`v1`)
and the second value of `args` as the third argument to the objective function
(`v2`).

`args` is the solution to our problem of near-identical functions for different
x and y values - we can use `args` to pass the `x` and `y` values for our
function to work on:

```{python}
def rmse_any_line(c_s, x_values, y_values):
    c, s = c_s
    predicted = c + x_values * s
    errors = y_values - predicted
    return np.sqrt(np.mean(errors ** 2))
```

Now we can use the same objective function for any pair of `x` and `y` values:

```{python}
minimize(rmse_any_line, [2.25, 0.47], args=(easiness, quality))
```

```{python}
minimize(rmse_any_line, [2.25, 0.47], args=(easiness, clarity))
```
