OCR Python Assignment Tasks

Task 1 : Compressing text

Develop a program that analyses a sentence that contains several words without punctuation. When a word in that sentence is input, the program identifies all of the positions where the word occurs in the sentence. The system should not be case sensitive: Ask, ask, ASK should be treated as the same word.

For example, in the sentence

"ASK NOT WHAT YOUR COUNTRY CAN DO FOR YOU ASK WHAT YOU CAN DO FOR YOUR COUNTRY"

The word ‘COUNTRY’ occurs in the 5th and 17th positions.

Analyze the requirements for this system and design, develop, test and evaluate a program to locate and return the position(s) of the word you have selected in a particular sentence or return an error message if the word is not in the sentence.

Task01 Description:

Please contact me at immukesh72@gmail.com for further assistance on such kind of assignments

Task 2 : Identifying Indivisual words in a sentence

Develop a program that identifies individual words in a sentence, stores these in a list and replaces each word in the original sentence with the position of that word in the list.
For example, the sentence

"ASK NOT WHAT YOUR COUNTRY CAN DO FOR YOU ASK WHAT YOU CAN DO FOR YOUR COUNTRY"
contains the words ASK, NOT, WHAT, YOUR, COUNTRY, CAN, DO, FOR, YOU
The sentence can be recreated from the positions of these words in this list using the sequence
1,2,3,4,5,6,7,8,9,1,3,9,6,7,8,4,5

Save the list of words and the positions of these words in the sentence as separate files or as a single file.
Analyse the requirements for this system and design, develop, test and evaluate a program to:
• identify the individual words in a sentence and store them in a list
• create a list of positions for words in that list
• save these lists as a single file or as separate files.


Please contact me at immukesh72@gmail.com  for further assistance on such kind of assignments.

Task 3 : Compress The Text File With Several Sentences

Develop a program that builds upon the technique from Task 2 to compress a text file with several sentences, including punctuation. The program should be able to compress a file into a list of words and list of positions to recreate the original file. It should also be able to take a compressed file and recreate the full text, including punctuation and capitalisation, of the original file.

Analyse the requirements for this system and design, develop, test and evaluate a program to compress a text file and reproduce the original text from a compressed file. You will need to create a text file with more than one sentence to test your system.

Requirements: Develop a program that builds upon the technique from Task 2 to compress a text file with several sentences, including punctuation. The program should be able to compress a file into a list of words and list of positions to recreate the original file. It should also be able to take a compressed file and recreate the full text, including punctuation and capitalization, of the original file.

Steps to develop above program:

Please contact me at immukesh72@gmail.com for further assistance on such kind of assignments

How to connect remote linux server from local linux server with ssh

Using the SSH protocol and by sharing the public key, you can connect and authenticate to remote servers and services. With SSH keys, you can connect to remote server from local server.
Let us consider we have 2 linux machine local server and remote server.

Step 1 . Check if you have already key pair 

Before starting let check if you have public and private in your local server for the user local-user.

[local-user@ip-xx-xx-xx ~]$ ls -l ~/.ssh
-rw-------. 1 local-user local-user  401 Nov 19 09:13 authorized_keys
-rw-------. 1 local-user local-user 3243 Nov 19 17:36 id_rsa
-rw-r--r--. 1 local-user local-user  743 Nov 19 17:36 id_rsa.pub
-rw-r--r--. 1 local-user local-user  803 Nov 19 17:32 known_hosts

If you able to see id_rsa and id_rsa.pub then that means you have already generated key pair otherwise you need to generate key pair.

Step 2. Generate the public and private key pair

Run the following command to generate the ssh key pair.

[local-user@ip-172-31-52-47 ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/local-user/.ssh/id_rsa):[ Hit Enter key ]
Created directory '/home/local-user/.ssh'.
Enter passphrase (empty for no passphrase):[ Hit Enter key ]
Enter same passphrase again:[ Hit Enter key ]
Your identification has been saved in /home/local-user/.ssh/id_rsa.
Your public key has been saved in /home/local-user/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:Pz5p8KbvF/kcgOTSak45/ZGkE5saGVLZD36U1okLtg4 local-user@ip-172-31-52-47.ec2.internal
The key's randomart image is:
+---[RSA 2048]----+
|         o   + . |
|        o ** = o |
|       . O .     |
|      . E B *    |
|       .SX xB +  |
|        X.B = .  |
|       + *o+ = . |
|        o.*.o o  |
|        .*+o     |
+----[SHA256]-----+

Now you will be able to see public and private key in the /home/local-user/.ssh/ directory.

Step 3 : Share the public key to remote server

Copy and install the public key using ssh-copy-id command

$ ssh-copy-id -i remote-user@<ip-address-of-remote-server>
    remote-user@<ip-address-of-remote-server> password:
   

Note : You can also direct copy the content of public key and pate into authorized_keys (~/.ssh/authorized_keys) file of remote server.
 
Step 4. Access remote server with SSH

Now try logging into the machine, with "ssh remote-user@<ip-address-of-remote-server>" from  local server with local-user.

******************** END ********************

5 ways to count the lines of a file in Linux

There are multiple ways to count the number of lines of a file in Linux. In our daily life we need to count number of lines of a csv file, text file etc and the most popular command we use "wc -l". In this article i will show you 5 different ways to find number of line along with wc -l

Let us consider we have 2 sample file name as sample1.txt and sample2.txt which having 10 and 3 lines as following. I will show you different examples to get the number of lines.

Contents of sample1.txt

[~]$ cat sample1.txt

One
Two
Three
four
five
six This is longest line
seven
eight
nine
ten

Contents of sample2.txt

[~]$ cat sample2.txt
One
Two Longest Line
Three

The wc (word count) command is very popular in Unix/Linux to find number of lines count, word counts, byte and characters count in a file. The syntax of wc command is as following.

wc  [OPTION]... [FILE]...

Where OPTION are as below:

  -c, --bytes            print the byte counts
  -m, --chars            print the character counts
  -l, --lines            print the newline counts
      --files0-from=F    read input from the files specified by
                           NUL-terminated names in file F;
                           If F is - then read names from standard input
  -L, --max-line-length  print the length of the longest line
  -w, --words            print the word counts

[~]$ wc sample1.txt sample2.txt

10 14 70 sample1.txt
 3  5 27 sample2.txt
13 19 97 total

In the above example first column shows number of line , 2nd column shows words and 3rd column shows number of chars. Last row shows the total counts of all files.

Lets us have a look of different ways to find the number of lines in a file , we will use sample1.txt in our demo.

1) Counting lines with WC command

[~]$ cat sample1.txt | wc -l
10

or

[~]$ wc -l sample1.txt
10 sample1.txt

2) Counting lines with sed command
sed uses "=" operator for line numbering and "$" gives the last count of numbering which is total number of lines.

[~]$ sed -n "$=" sample1.txt
10

3) grep command with -c option:

To count all non-empty lines or non-blank line.

[~]$ grep -c "." sample2.txt
3

To count all line including blank or empty lines.

[~]$ grep -c ".*" sample2.txt
4

or

[~]$ grep -c "^" sample2.txt
4

4). Counting line with awk:

awk with NR variable gives the line numbers and by printing NR with the end block it gives the line number of the last line which is nothing but a total number of lines in file.

[~]$ awk 'END {print NR}' sample1.txt
10

5). Counting lines with perl:

End block as in awk we can use also with perl. In Perl "$." gives the number of lines.

[~]$ perl -lne 'END {print $.}' sample1.txt
10

Note:

To find the longest line character count we can use wc with -L option. As shown below in sample1.txt file the longest line having 24 character where as in sample2.txt it is 16.

[~]$ wc -L sample1.txt sample2.txt
24 sample1.txt
16 sample2.txt
24 total

Have a look on couple of heck of counting lines.

Trick 01:

cat -n sample1.txt | tail -n 1 | cut -f1

Explanation : "cat -n sample1.txt" keeps a number line in file and pipe "tail -n1" gives the first row from bottom which includes line number and last line content. Then pipe "cut -f1" take out the first field of last line(last line number) which is nothing but the total count of lines.

Trick 02:

Using the while loop reading file line by line and increasing counter.

#!/bin/bash

count=0
while read
do
  ((count=$count+1))
done <sample1.txt

echo $count

***End***

Split command in Linux/Unix

Split command is very useful when you are managing large file . Consider you have a csv file with millions of records and its taking too much time to open. In this case we can split file into small pieces and can access it easily in any GUI.

The default size for each split file is 1000 lines, and default PREFIX is "x". However we can split file based of number of lines or bytes and can change the prefix as well. In this article i will show you how to use split command with examples.

Let us consider we have a file testfile.csv with 1342 records.

[~]$ cat testfile.csv | wc -l
1342

1). Split simple example :

As you can see below split command split file testfile.csv in 2 pieces with default prefix x. testfile.csv file having total 1342 records hence by default it split first file name as xaa with default 1000 line and second file name as xab with remaining records 342.

[~]$ split testfile.csv

[~]$ ls
testfile.csv  xaa  xab

[~]$ cat xaa | wc -l
1000

[~]$ cat xab | wc -l
342

2) Split file with specific number of lines:

We can use -l option with split command to achieve specific number of line into split files. Let us we want to split file with 500 records for each then use following command.

[~]$ split -l 500 testfile.csv

[~]$ ls
testfile.csv  xaa  xab  xac

[~]$ cat xaa | wc -l
500

[~]$ cat xab | wc -l
500

[~]$ cat xac | wc -l
342

3) Split file with a specific prefix:

If we want to use our own prefix  "NEW" in split files use the following command.

[~]$ split -l 500 testfile.csv NEW

[~]$ ls
NEWaa  NEWab  NEWac  testfile.csv

4) Split file with numeric suffix:

We can append our own numeric suffix like 00,01,02... instead default xa,xb,xc .... with -d option as following.

[~]$ split -l 50 -d testfile.csv NEW

[~]$ ls
NEW00  NEW02  NEW04  NEW06  NEW08  NEW10  NEW12  NEW14  NEW16  NEW18  NEW20  NEW22  NEW24  NEW26
NEW01  NEW03  NEW05  NEW07  NEW09  NEW11  NEW13  NEW15  NEW17  NEW19  NEW21  NEW23  NEW25  testfile.csv

By default numeric suffix has 2 digits and you may need to increase the number of digits if split files crossing more than 100 files. In that case you will get following "suffixes exhausted" message and you may loose some split files after NEW99.

[~]$ split -l 10 -d testfile.csv NEW
split: output file suffixes exhausted

[~]$ ls
NEW00  NEW05  NEW10  NEW15  NEW20  NEW25  NEW30  NEW35  NEW40  NEW45  NEW50  NEW55  NEW60  NEW65  NEW70  NEW75  NEW80  NEW85  NEW90  NEW95  testfile.csv
NEW01  NEW06  NEW11  NEW16  NEW21  NEW26  NEW31  NEW36  NEW41  NEW46  NEW51  NEW56  NEW61  NEW66  NEW71  NEW76  NEW81  NEW86  NEW91  NEW96
NEW02  NEW07  NEW12  NEW17  NEW22  NEW27  NEW32  NEW37  NEW42  NEW47  NEW52  NEW57  NEW62  NEW67  NEW72  NEW77  NEW82  NEW87  NEW92  NEW97
NEW03  NEW08  NEW13  NEW18  NEW23  NEW28  NEW33  NEW38  NEW43  NEW48  NEW53  NEW58  NEW63  NEW68  NEW73  NEW78  NEW83  NEW88  NEW93  NEW98
NEW04  NEW09  NEW14  NEW19  NEW24  NEW29  NEW34  NEW39  NEW44  NEW49  NEW54  NEW59  NEW64  NEW69  NEW74  NEW79  NEW84  NEW89  NEW94  NEW99

To overcome this you can increase number of digits in suffix by using -a option as following.

[~]$ split -l 10 -a 3 -d testfile.csv NEW

[~]$ ls
NEW000  NEW007  NEW014  NEW021 .........  NEW099  NEW100  NEW101  .........  NEW132

5) Split file with 4000 bytes output:

We can use -b option with desired number of size.

[~]$ split -b4000 testfile.csv
              (or)
[~]$ split -b4k testfile.csv

[~]$ ls -ltr x*
-rw-rw-r-- 1 mukesh mukesh 3888 Oct  7 21:14 xae
-rw-rw-r-- 1 mukesh mukesh 4096 Oct  7 21:14 xad
-rw-rw-r-- 1 mukesh mukesh 4096 Oct  7 21:14 xac
-rw-rw-r-- 1 mukesh mukesh 4096 Oct  7 21:14 xab
-rw-rw-r-- 1 mukesh mukesh 4096 Oct  7 21:14 xaa

6) Split file with 2 files of equal length:

We can use -n option in place of -l as following to achieve specific number of file of same records.

[~]$ split -n 2 -d testfile.csv NEW

[~]$ ls
NEW00  NEW01  testfile.csv

[~]$ cat NEW00 | wc -l
670

[~]$ cat NEW01 | wc -l
672

[~]$ cat testfile.csv | wc -l
1342

In above example the expected count should be 671 into each NEW00 and NEW01 but its not. If anyone could explain me it would be appreciated.

***End***