Web Crawler:

In this project I have crawled the website http://lums.edu.pk/ till the depth of 10 pages from one page. I have developed the assignments in two parts; 

-First one is URLsFetcher which fetches all the urls and their html content and saves data in 2 separate files in D drive, keeping the index common for URL and its content in corresponding file(I have used this technique to make the search faster using basic form of indexing).

-Second is the application interface in which user can enter query to be searched and it parses the content txt files to get the results and it saves the searched queries persistently(you can re open the application and all the previously searched term would be in suggestive drop down using auto-complete). We can update the txt files by running the URLsFetcher again and web crawler part will work on it as well. I have used multi-threading for searching and content writing

PS: I have attached screenshot of app, code both programs and all txt files. Technology and IDE used: C#,Windows Form Application, VS2012.

PDF: https://goo.gl/n6vbS3
Project Files: https://goo.gl/fyrXd1

Distributed File System Prototype:

In this project I have used asynchronous sockets to enable multiple clients to connect to a server. I have developed the following functionalities of DFS assignment. 

1. CreateFile(char*filename): Server returns 1 if a new file is created in its local directory on the behalf of user’s request. Filename can be up to 30 characters. Server returns -1 if a local copy of same filename already exists or there is some other problem.

 2. OpenFile (char *filename, int mode): Opening a file means creating a local copy of the file at client’s end and client performs read and write operations on it. Mode can be “read” i.e. 0 or “write” i.e. 1. The system only allows 1 client to open a file in write mode at a given time. If call is successful, the call returns a positive value i.e. file descriptor. This file descriptor is subsequently used to perform read/write operations. Server returns -1 if file doesn’t exist or -2 if file has already been opened in write mode by any other client. 

3. ReadFile (int fd, char *buffer): This call reads the contents of file described by its file descriptor and fills the buffer with it. The contents should be printed on client’s screen. 

4. WriteFile(int fd, char *buffer, int length, int mode): This call writes the contents of buffer in a file described by its file descriptor “fd” up to the given length. Writes can happen in 3 different modes. i. Blocking: In this mode, file will be written to server and the write update is sent to all the clients who have opened this file in read mode. We call such clients as active replicas. Once the server and all replicas have been updated, call to this function returns with 1. ii. Non-Blocking: In this mode, file will be written to the server and it returns with a value = 1. That means, only the server has been updated and all the subsequent replicas are yet to be updated who have opened the file in read mode. iii. Disconnected: In this mode, the file is written on the local directory of the client. This is suitable for a mobile client who may not be connected to the server all the time. The server and replicas are only updated when the client closes the file. Technology Used: I have used mono and Xamarin as IDE for this assignment and language used is C#.

Here are the links for assignment description and projects files.

PDF: https://goo.gl/A8uDCt

Project File: https://goo.gl/nXRsPN