How to install Tensorflow GPU in windows

I have already briefed about tensorflow in my old blogs, in short it is an open-source library with is capable of running machine learning algorithms. Now lets jump to the installation part.

*Installation of Tensorflow with GPU support and anaconda*

Hardware requirements

NVIDIA GPU card with CUDA® Compute Capability 3.5 or higher, CUDA-enabled GPU cards

Software requirements

The following NVIDIA software must be installed on your system:

• • • Visual Studio is required, so download the latest version of Visual studio code
    • NVIDIA® GPU drivers —CUDA 10.0 requires 410.x or higher.
      • Download your driver accordingly matching your GPU (Nearly 400MB)
    • CUDA® Toolkit —TensorFlow supports CUDA 10.0 (TensorFlow >= 1.13.0)
      • • Choose the latest version (CUDA Toolkit 10.1)
        • Choose Platform as windows and download for your OS
        • Choose Installer Type as exe(local)
        • Click Download (Base Installer). It is going to take 1.3gb space.
        • After Downloading, open start and type environment variable
        • In the Pop-Up box click environment variable
        • In the system variable locate PATH and choose edit after selecting PATH
        • Select NEW (after you choose the edit option for PATH)
        • In the box presented to you after clicking new type

          C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin
          C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\libnvvp

    • cuDNN SDK (>= 7.4.1)
      • To download this you need to have an account so sign up by clicking join
      • After making an account, Agree to all terms
      • Download cuDNN v7.6.0 (Latest on your screen)
      • Download according to your operating system(around 200MB)
      • After downloading the zip file, unzip and keep it in your C drive
      • Now open environment variable by typing it in your start menu
      • In the Pop-Up box click environment variable
      • In the system variable locate PATH and choose edit after selecting PATH
      • Select NEW (after you choose the edit option for PATH)
      • C:\cuda\bin Add this to you NEW environment variable
    • (Optional) TensorRT 5.0 to improve latency and throughput for inference on some models.

NOTE: You can skip the last two step if you want as its not necessary.

Installing with Anaconda

• • • Install Anaconda from there.
    • After the installation, open a Command Prompt and type conda create -n tensorflow
    • After this gets over we can now activate tensorflow by typing activate tensorflow
    • Run the following command to install it completely

      • pip install tensorflow-gpu

We are in the ENDGAME now

To check everything works fine open python interpreter and type import tensorflow​.

import tensorflow as tf

a = tf.constant(50,name="a")
b = tf.constant(25,name="b")
c = tf.add(a,b,name="c")

sess = tf.Session()
output = sess.run(c);
print(output)
sess.close()

If it compiles without any errors VOILA!

NOTE: If you face errors then you need to search them on StackOverflow.

 

Booting Fedora ARM image to Raspberry PI

Introduction to Raspberry PI

It is a small single board computer which can perform all basic functions which a laptop or a Desktop PC can do. It even has its own OS(Operating system). If we want we can change it and boot our favorite OS.

In this blog you are going to learn how to boot Fedora ARM in a SD-Card and boot up Fedora.

Installing fedora ARM

For the 1st step, you need to download Etcher(a free and open-source utility used for burning image files such as .iso and .img files)

Now Download the Fedora ARM (Your favorite version). I have installed Fedora 29

Now, Open etcher and select your ARM by clicking on Select Image

After this You need to select you drive(Your SD Card) where you need to burn the image by clicking Select Drive

Now, wait Patiently and when its done(It says Flash Complete) you need to put you SD card back to your slot and then Boot up Fedora.

It will continue with the boot where you need to setup a Username and password. After sometime you should be welcomed by Fedora.

Core Libraries.

1. Pandas 

Pandas is a Python package designed to do work with “labeled” and “relational” data simple and intuitive. Pandas is a perfect tool for data wrangling. It designed for quick and easy data manipulation, aggregation, and visualization.

There are two main data structures in the library:

“Series” – 1 dimensional

“Data Frames”, 2 dimensional

 For example, when you want to receive a new Dataframe from these two types of structures. As a result, you will receive such Df by appending a single row to a DataFrame by passing a Series:

 Here is just a small list of things that you can do with Pandas:

• Easily delete and add columns from DataFrame

• Convert data structures to DataFrame objects

• Handle missing data, represents as NaNs

• Powerful grouping by functionality

2. Matplotlib 

Another SciPy Stack core package and another Python Library that is tailored for the generation of simple and powerful visualizations with ease is Matplotlib. It is a top-notch piece of software which is making Python (with some help of NumPy, SciPy, and Pandas) a cognizant competitor to such scientific tools as MatLab or Mathematica. However, the library is pretty low-level, meaning that you will need to write more code to reach the advanced levels of visualizations and you will generally put more effort, than if using more high-level tools, but the overall effort is worth a shot.

With a bit of effort you can make just about any visualizations:

• Line plots;

• Scatter plots;

• Bar charts and Histograms;

• Pie charts;

• Stem plots;

• Contour plots;

• Quiver plots;

• Spectrograms.

There are also facilities for creating labels, grids, legends, and many other formatting entities with Matplotlib. Basically, everything is customizable.

The library is supported by different platforms and makes use of different GUI kits for the depiction of resulting visualizations. Varying IDEs (like IPython) support functionality of Matplotlib.

There are also some additional libraries that can make visualization even easier.

Booting Fedora from a PXE server

A short introduction on PXE

PXE(Preboot Execution Environment) allows a workstation to boot from a server on a network to boot the operating system on the local drive.A PXE-enabled workstation connects its NIC (Network Interface Controller)to the LAN via a Jumper, which keeps the workstation connected to the network even when there is no power.

Booting 

THE SERVER

• 1st we need to disable SELinux(Security-Enhanced Linux)

  • sudo setenforce 0

• After disabling SELinux, let’s install Cobbler

  • sudo dnf install cobbler dnf-plugins-core pykickstart yum-utils

• When the installation gets complete edit the cobbler configuration file located at /etc/cobbler/settings​. We need to tweak the file and add the result of openssl passwd -1 ​.

• For example,

  • default_password_crypted: "<your-result>"

• Now, lets configure the listening server, by adding your Private IP to the line

  • next_server: 192.168.x.y

  • server: 192.168.x.y

• Lets now configure the DHCP located at /etc/cobbler/dhcp.template.

  • subnet 192.168.x.x netmask 255.255.255.0 {
         option routers             192.168.x.y;
         option domain-name-servers 192.168.x.y;
         option subnet-mask         255.255.255.0;
         range dynamic-bootp        192.168.x.100 192.168.x.254;
         default-lease-time         21600;
         max-lease-time             43200;
         next-server                $next_server;

• Go back to /etc/cobbler/settings,  and change manage_dhcp to 1

  • manage_dhcp: 1

• Now lets download bootloader

  • sudo cobbler get-loaders

• We need to restart the services

  • systemctl start cobblerd.service
    systemctl start httpd
    sudo cobbler sync

• We need to now transfer the Fedora ISO to Cobbler server and mount it

  • sudo mount -t iso9660 -o loop,ro /path/of/image/Fedora-Workstation-Live-x86_64-29-1.2.iso /mnt

• Now we import the ISO file to the Cobbler server

  • sudo cobbler import --name=fedora29 --arch=x86_64 -breed=redhat --os-version=fedora29 --path=/mnt

THE CLIENT

• Now we need to enter the BIOS(Basic Input Output system) by pressing the specific key to enter the BIOS(For Example: F1, Ecs, F10, F12)
• Then navigate to Boot options and change the priority to PXE Boot or Network Adapter Boot.
• Save your settings and you are done. You will be successfully booting Fedora 29 from a PXE Server.

Running Tensorflow on Fedora

What is Tensorflow?

So, lets start with what is tensorflow?

Tensorflow is an open-source library with is capable of running machine learning algorithms.It was developed by google brain’s team(Second-gen, DistBelief was the 1st).

Now-a-days it is widely used by many companies, Machine Learning algorithmist, Students, teachers, etc.This library can be used in python mainly. Some interfaces are available to execute and construct graph in C++, Java and Go. Tensorflow also has a JavaScript library.

NOTE: You need to have python3.6 installed (sudo dnf install python3)

Installation of TensorFlow CPU in Fedora

• In the 1st stage, we need to create a virtual environment(we will use virtualenv [isolated Python environment])pip3 install virtualenv
• Now let’s create a virtual environment(virtualenv).

  • virtualenv --system-site-packages -p /usr/bin/python3.6 ./venv && source ./venv/bin/activate

• Now lets install tensorflow

  • pip install tensorflow

• Now, to move out of the virtual environment type deactivate.

Now lets test our TensorFlow

• Create a file,

touch mark1.py
nano mark1.py

• Then write this in the mark1.py file,

import tensorflow as tf

class SquareTest(tf.test.TestCase):
    def testSquare(self):
      with self.test_session():
          x = tf.square([2, 3])
          self.assertAllEqual(x.eval(), [4, 9])

if __name__ == '__main__':
    tf.test.main()

 

** If you want to learn how to install tensorflow GPU, Please read this

Running OpenCV on Fedora

What is openCV?

OpenCV (Open Source Computer Vision) is a library of programming functions mainly which aims at real-time computer vision. It has C++, Python, Java and MATLAB interfaces and it supports Windows, Linux, Android and Mac OS.

It can detect and recognize faces, identify objects, identify human actions in videos, track camera movements, track moving objects, stitch images together to produce a high resolution image of an entire scene, find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, etc.

How to install openCV?

1st Method

Install all packages with following command in terminal as root.(the screenshot aslo contains numpy, Please ignore it)

sudo dnf install opencv        # For C++


OR

You can also do this

dnf install python3-opencv        # For Python

OR

2nd Method

Install from PyPi store

pip3 install opencv-python        # For Python x86

3rd Method(From Source)

• 1st we need to install the dependencies before we need to start, type this in you terminal:

• • sudo dnf install cmake-gui ffmpeg-devel libpng-devel libjpeg-turbo-devel jasper-devel libtiff-devel tbb-devel eigen3-devel

• Now we need to download openCV

• • yum install git

• • git clone https://github.com/Itseez/opencv.git

• Now change directory to the downloaded file(opencv) and Create a new build folder

• • mkdir build

• • cd build

• Now, lets move to the most important part, the installation part. Installation has to be configured using cmake(specifies which modules are to be installed, installation path, which additional libraries to be used).

• • cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..

• (optional) To install TBB and Eigen

• • cmake -D WITH_TBB=ON -D WITH_EIGEN=ON ..

• To Enable documentation and disable tests and samples, type this

• • cmake -D BUILD_DOCS=ON -D BUILD_TESTS=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_EXAMPLES=OFF ..

• Let’s disable all GPU modules

• • cmake -D WITH_OPENCL=OFF -D WITH_CUDA=OFF -D BUILD_opencv_gpu=OFF -D BUILD_opencv_gpuarithm=OFF -D BUILD_opencv_gpubgsegm=OFF -D BUILD_opencv_gpucodec=OFF -D BUILD_opencv_gpufeatures2d=OFF -D BUILD_opencv_gpufilters=OFF -D BUILD_opencv_gpuimgproc=OFF -D BUILD_opencv_gpulegacy=OFF -D BUILD_opencv_gpuoptflow=OFF -D BUILD_opencv_gpustereo=OFF -D BUILD_opencv_gpuwarping=OFF ..

• Now lets set the installation path and build type

• • cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..

• If all goes well you will get this,

• Now let’s build the files using make command and install it using make install command. make install should be executed as root(sudo).

• • make

• • su

• • make install

• The installation process is over,  All files are installed in /usr/local/.

Configure openCV to run with python

• Move the module to any folder in Python Path

• • su mv /usr/local/lib/python3.6/site-packages/cv2.so /usr/lib/python3.6/site-packages

• Add /usr/local/lib/python2.7/site-packages to the PYTHON_PATH

• • export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.6/site-packages

Test the installation

Open up a terminal and type pythonand in the python interpreter type import cv2, it should import without any errors.

Automation of a boring job

Automation

Everyday humans evolve and try to make their job as simple as possible. Exactly, the main aim of ansible is to help programmers to automate their job.

App server setup

We all know that a lot of time is taken while setting up a app server, even after knowing the setup prerequisites like installation of packages and then verification takes up time.

Ansible is a great tool used in automation in many areas like software deployment, Configuration management, etc.

Steps of installing a app server

There are some major steps while setting up a app server:

• Installation of packages like git, apache, mysql
  • We used to install these packages one by one,

    • sudo dnf install git
      sudo dnf install httpd
      #we need to allow apache through firewall
      sudo firewall-cmd --permanent --add-port=80/tcp
      sudo firewall-cmd --permanent --add-port=443/tcp
      # Reload
      sudo firewall-cmd --reload
      sudo dnf install mysql
      sudo dnf install python-devel
      sudo dnf install mysql-devel

• Downloading the project from git

  • git clone https://github.com/username/yourrepo.git

• Creating a virtual environment for the project and installation of the prerequisites inside the virtual lab

  • pip install virtualenv
    pip install mysql-python

• Creating database

  • mysql> CREATE DATABASE TABLE;
    mysql> USE TABLE
      Database changed
    shell> mysql -h HOST -u user -p TABLE
    Enter password: ********
    
    

• Transfer of files
• Configuration of the server settings

  • chmod 777 file
    #installing the requirements of the virtual environment
    pip install -r requirements.txt

 

 

 

Running Tensorflow GPU on Fedora

What is Tensorflow?

So, lets start with what is tensorflow?

• Tensorflow is an open-source library with is capable of running machine learning algorithms.
• It was developed by google brain’s team(Second-gen, DistBelief was the 1st).
• Now-a-days it is widely used by many companies, Machine Learning algorithmist, Students, teachers, etc.
• This library can be used in python mainly. Some interfaces are available to execute and construct graph in C++, Java and Go. Tensorflow also has a JavaScript library(TensorFlow.js)
• TensorFlow supports CUDA(CUDA is a parallel computing platform and API model created by Nvidia.). It helps to raise the performance of our program.

Installation of TensorFlow GPU in Fedora

• In the first step we need to check that we have a NVIDIA GPU:

sudo lspci | grep -i NVIDIA    #information about all PCI buses and devices in the computer. 

(In a line it mentions about your VGA adapter)

• Now we need to add negativo repo:

dnf config-manager --add-repo=http://negativo17.org/repos/fedora-nvidia.repo

Then we need to install the NVIDIA driver, and the necessary libraries for cuda operations.

• Now we need to complete the process to install CUDA and NVIDIA drivers we need to type:

  • dnf install kernel-devel dkms-nvidia  nvidia-driver-cuda
    dnf install cuda-devel cuda-cudnn-devel

• After the process completes reboot your computer.
• After rebooting type this

  • sudo lsmod  | grep nv

  • This will show somting like this:

    • nvidia_drm             45056  0
      nvidia_modeset        901120  1 nvidia_drm
      nvidia_uvm            684032  0
      nvidia              13914112  2 nvidia_modeset,nvidia_uvm
      drm_kms_helper        159744  2 i915,nvidia_drm
      drm                   352256  5 i915,nvidia_drm,drm_kms_helper

• Now we move on to the main step, Installation of TensorFlow for the GPU:
  • Install the TensorFlow build, which is built to be ready for the latest versions(python 3.x)

    • virtualenv --system-site-packages -p python3 ~/env3
      . ~/env3/bin/activate
      
      
      pip install tf-nightly-gpu

• Now we will test that our installation works or not:

  • import tensorflow as tf
    
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)
    
    MySession = tf.Session(config=tf.ConfigProto(log_device_placement=True))
    print(MySession.run(c))

 

NOTE

If it shows  /dev/nvidia0 does not exist then you’re not running on the Nvidia card as a display adapter, or you have not installed the nvidia-modprobe.

Learn about YAML files.

What is YAML?

YAML is a recursive acronym for “YAML Ain’t Markup Language!” is a data serialization language which is easily read by ‘humans’. It’s commonly used in configuration files(Not limited to it).

NOTE: JSON, A YAML parser can understand JSON, but a JSON parser can not understand YAML.

The file extension is either .yml or  .yaml.

Syntax

Now let’s jump to the syntax-

• Every YAML document starts with 3 hyphens ---
• Every YAML document ends with 3 dots ...
• YAML is case-sensitive
• YAML doesn’t support the use of tabs. It supports spaces
• Data types of yaml:
  • Sequence:- Lists
  • Scalar:- Numbers
  • Mappings:- Dictionary
• Comments are made by the hash symbol (#). For example:

  • # This is a comment. You can explain your code here

• The components of the list is denoted by a hyphen (-)
• Basic syntax:

  • ---
    CLASS-ROLL: 1519182      # Class Roll-No
    Name: "Nishant Parhi"    # Name of student
    Pass: true               # Statement is true
    6th subject: null        # This is null

• About dictionaries:
  • How to write dictionaries in 2 ways:

    • # About Me
      Nishant:
          name: Nishant Parhi
          Age: 17
          Class: 12

    • ---
      Nishant: {name: Nishant Parhi, age: 17, Class: 12}

• About lists:
  • All components in a list begin with the indentation level starting with a "- " (a hyphen and a space). For example:

    • ---
      # List of favorite games
      games:
          - GTA-5
          - Watch Dogs 2
          - FIFA 18
          - Mario Run
      ...

• About Multi-Line stings
  •  There are two ways to write multi-line strings, using | character and > character.

    • data: |
         I am learning about YML files
         I am finding it interesting
            It is easy to learn
             "You can also learn it"

    • data: >
         This is
         another method
         of writing multi-string
      
         Blank lines create a
         paragraph break