{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "What is PyTorch?\n",
    "================\n",
    "\n",
    "It’s a Python-based scientific computing package targeted at two sets of\n",
    "audiences:\n",
    "\n",
    "-  A NumPy replacement that can leverage the power of GPUs\n",
    "-  a deep learning research platform \n",
    "\n",
    "\n",
    "Tensors\n",
    "---------------\n",
    "\n",
    "Tensors are similar to NumPy’s ndarrays, with the added benefit of the use of a GPU to accelerate computation speed.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct a 5x3 matrix, uninitialized:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.zeros(5, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct a randomly initialized matrix:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.4898, 0.7586, 0.4739],\n",
      "        [0.4444, 0.5913, 0.8829],\n",
      "        [0.2168, 0.1519, 0.7091],\n",
      "        [0.3477, 0.4247, 0.7335],\n",
      "        [0.2396, 0.5724, 0.7993]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.rand(5, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct a tensor  from data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([5.5000, 3.0000])\n"
     ]
    }
   ],
   "source": [
    "x = torch.tensor([5.5, 3])\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or create a tensor based on an existing tensor. These methods\n",
    "will reuse properties of the input tensor:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]])\n",
      "tensor([[ 0.7564,  0.9491, -0.5533],\n",
      "        [-0.2931, -0.2400,  0.6996],\n",
      "        [ 0.9043, -0.8304,  1.6302],\n",
      "        [-1.0803, -0.8668,  0.6330],\n",
      "        [-1.4586,  0.0956,  0.4361]], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "# new_* methods create a tensor of the same type:\n",
    "x = x.new_ones(5, 3)\n",
    "print(x)\n",
    "\n",
    "# create a tensor like x and override dtype:\n",
    "x = torch.randn_like(x, dtype=torch.double) \n",
    "print(x)  # result has the same size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get its size:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([5, 3])\n"
     ]
    }
   ],
   "source": [
    "print(x.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that ``torch.Size`` is in fact a tuple, so it supports all tuple operations.\n",
    "\n",
    "### Operators\n",
    "\n",
    "**Addition:**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.3167,  1.1979,  1.2882],\n",
      "        [ 0.7474, -1.4148,  0.4302],\n",
      "        [ 1.8626,  1.9702,  0.0323],\n",
      "        [ 1.4492,  0.9571,  0.5144],\n",
      "        [-0.2637, -0.6164,  0.7562]])\n"
     ]
    }
   ],
   "source": [
    "y = torch.rand(5, 3)\n",
    "print(x + y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use the torch add function:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.3167,  1.1979,  1.2882],\n",
      "        [ 0.7474, -1.4148,  0.4302],\n",
      "        [ 1.8626,  1.9702,  0.0323],\n",
      "        [ 1.4492,  0.9571,  0.5144],\n",
      "        [-0.2637, -0.6164,  0.7562]])\n"
     ]
    }
   ],
   "source": [
    "print(torch.add(x, y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You also have the option of providing an output tensor as argument:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.3167,  1.1979,  1.2882],\n",
       "        [ 0.7474, -1.4148,  0.4302],\n",
       "        [ 1.8626,  1.9702,  0.0323],\n",
       "        [ 1.4492,  0.9571,  0.5144],\n",
       "        [-0.2637, -0.6164,  0.7562]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.3167,  1.1979,  1.2882],\n",
      "        [ 0.7474, -1.4148,  0.4302],\n",
      "        [ 1.8626,  1.9702,  0.0323],\n",
      "        [ 1.4492,  0.9571,  0.5144],\n",
      "        [-0.2637, -0.6164,  0.7562]])\n"
     ]
    }
   ],
   "source": [
    "result = torch.empty(5, 3)\n",
    "torch.add(x, y, out=result)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Addition: in-place\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.3167,  1.1979,  1.2882],\n",
       "        [ 0.7474, -1.4148,  0.4302],\n",
       "        [ 1.8626,  1.9702,  0.0323],\n",
       "        [ 1.4492,  0.9571,  0.5144],\n",
       "        [-0.2637, -0.6164,  0.7562]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.3167,  1.1979,  1.2882],\n",
      "        [ 0.7474, -1.4148,  0.4302],\n",
      "        [ 1.8626,  1.9702,  0.0323],\n",
      "        [ 1.4492,  0.9571,  0.5144],\n",
      "        [-0.2637, -0.6164,  0.7562]])\n"
     ]
    }
   ],
   "source": [
    "# adds x to y\n",
    "y.add_(x)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"><h4>Note</h4><p>Any operation that mutates a tensor in-place has an added ``_`` in the name.\n",
    "    For example: x.copy_(y), x.t_(), will change x.</p></div>\n",
    "\n",
    "You can use standard NumPy-like indexing as we have gotten used to:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 0.7548, -2.2326,  1.2403,  0.6962, -0.9840])\n"
     ]
    }
   ],
   "source": [
    "print(x[:, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Resizing: If you want to resize/reshape tensor, you can use ``torch.view``:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 4]) torch.Size([16]) torch.Size([2, 8])\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(4, 4)\n",
    "y = x.view(16)\n",
    "z = x.view(-1, 8)  # the size -1 is inferred from other dimensions\n",
    "print(x.size(), y.size(), z.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Torch documentation\n",
    "\n",
    "You can read all about pytorch tensors, including transposing, indexing, slicing, mathematical operations, linear algebra, random numbers, etc.,\n",
    "in the [pytorch documentation](https://pytorch.org/docs/stable/torch.html).\n",
    "\n",
    "\n",
    "### NumPy Bridge\n",
    "\n",
    "Converting a Torch Tensor to a NumPy array and vice versa is a easy.\n",
    "\n",
    "The Torch Tensor and NumPy array will share their underlying memory\n",
    "locations, and changing one will change the other.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1., 1., 1., 1., 1.])\n"
     ]
    }
   ],
   "source": [
    "a = torch.ones(5)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.  1.  1.  1.  1.]\n"
     ]
    }
   ],
   "source": [
    "b = a.numpy()\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See how the numpy array changes in value:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([2., 2., 2., 2., 2.])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([2., 2., 2., 2., 2.])\n",
      "[ 2.  2.  2.  2.  2.]\n"
     ]
    }
   ],
   "source": [
    "a.add_(1)\n",
    "print(a)\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All tensors on  CPU except a CharTensor support converting to NumPy and back.\n",
    "\n",
    "### CUDA Tensors\n",
    "\n",
    "Tensors can be moved onto any device using the ``.to`` method.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    y = torch.ones_like(x, device=device)  # directly create a tensor on GPU\n",
    "    x = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    z = x + y\n",
    "    print(z)\n",
    "    print(z.to(\"cpu\", torch.double))       # ``.to`` can also change dtype together!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor(1., requires_grad=True)\n",
    "w = torch.tensor(2., requires_grad=True)\n",
    "b = torch.tensor(3., requires_grad=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build a computational graph.\n",
    "y = w * x + b\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "y.backward()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(2.)\n",
      "tensor(1.)\n",
      "tensor(1.)\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(x.grad)    # x.grad = 2 \n",
    "print(w.grad)    # w.grad = 1 \n",
    "print(b.grad)    # b.grad = 1 \n",
    "print(y.grad)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}