18_EffectOfWeights.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Effect of weights in the clustering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we know that 200 is good number of clusters, it is better to use real ideal and nadir values of the problem (in stead of ones attained from clustered results) and that it would be more justified to use stand closest to centroid as the representative of the cluster (even though the results are worse than if using the cold and mean mean of the cluster).\n",
    "\n",
    "Then the last thing not tried yet, is changing the weights of different clusters in the optimization phase. This far the weighting has been based on the number of stands in a cluster, but it would be better to scale the sum of all weights to 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    " %matplotlib inline\n",
    "import seaborn\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "from ASF import ASF\n",
    "from gradutil import *\n",
    "from pyomo.opt import SolverFactory\n",
    "seedn = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First lets take all the data in"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "revenue, carbon, deadwood, ha = init_boreal()\n",
    "n_revenue = nan_to_bau(revenue)\n",
    "n_carbon= nan_to_bau(carbon)\n",
    "n_deadwood = nan_to_bau(deadwood)\n",
    "n_ha = nan_to_bau(ha)\n",
    "ide = ideal(False)\n",
    "nad = nadir(False)\n",
    "opt = SolverFactory('cplex')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = pd.concat((n_revenue, n_carbon, n_deadwood, n_ha), axis=1)\n",
    "x_stack = np.dstack((n_revenue, n_carbon, n_deadwood, n_ha))\n",
    "#Normalize all the columns in 0-1 scale\n",
    "x_norm = normalize(x.values)\n",
    "x_norm_stack = normalize(x_stack)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cluster the data to some clusters and calculate correponding weights using both ways"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time \n",
    "nclust1 = 200\n",
    "c, xtoc, dist = cluster(x_norm, nclust1, seedn, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "rng = range(nclust1)\n",
    "total_weight = len(x_norm)\n",
    "w_orig = np.array([sum(xtoc == i) for i in rng])\n",
    "w_scale = np.array([sum(xtoc == i)/total_weight for i in rng])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate new cluster centers selecting the stand closest to the centroid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "c_close = np.array([x_norm_stack[np.argmin(dist[xtoc == i])] for i in range(nclust1)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate solution for some reference using original weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "ref = np.array((ide[0], nad[1]+1, nad[2]+1, nad[3]+1))\n",
    "ASF_lambda = lambda x: ASF(ide, nad, ref, c_close, weights=x[0], scalarization=x[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "orig_asf   = ASF_lambda((w_orig, 'asf'));   res_orig_asf = opt.solve(orig_asf.model)\n",
    "orig_stom  = ASF_lambda((w_orig, 'stom'));  res_orig_stom = opt.solve(orig_stom.model)\n",
    "orig_guess = ASF_lambda((w_orig, 'guess')); res_orig_stom = opt.solve(orig_guess.model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate solution for some reference using the scaled weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "scale_asf   = ASF_lambda((w_scale, 'asf'));   res_scale_asf = opt.solve(scale_asf.model)\n",
    "scale_stom  = ASF_lambda((w_scale, 'stom'));  res_scale_stom = opt.solve(scale_stom.model)\n",
    "scale_guess = ASF_lambda((w_scale, 'guess')); res_scale_stom = opt.solve(scale_guess.model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_to_real_values(x_stack, scale_asf.model, xtoc) - model_to_real_values(x_stack, orig_asf.model, xtoc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_to_real_values(x_stack, scale_stom.model, xtoc) - model_to_real_values(x_stack, orig_stom.model, xtoc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_to_real_values(x_stack, scale_guess.model, xtoc) - model_to_real_values(x_stack, orig_guess.model, xtoc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Actually there should'n be any difference between results because the scaling is linear. \n",
    "\n",
    "As you can see, now the cplex optimizer is used. Before when using clpk optimizer there were differences in the ASF  results. Probably some numerical instabilites of the non-commercial user.\n",
    "\n",
    "Differences between solvers should be mentioned in the thesis!"
   ]
  }
 ],
 "metadata": {
  "git": {
   "suppress_outputs": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}