Posts

Showing posts from 2016

Convert String to Base64 in Python

Image
Code to convert string (from utf-8) to base64 encoding and base64 back to utf-8: import base64 si = input('\nenter text:\n') print('\nyou entered\n' + si) b64 = base64.b64encode(bytes(si, 'utf-8')) print('\nbase64 is\n' + str(b64)) b64_bs = bytearray(base64.b64encode(bytes(si, 'utf-8'))) print('\nbase64 byte stream is\n' + str(list(b64_bs))) so = base64.b64decode(b64).decode('utf-8', 'ignore') print('\nstring is\n' + so) This example encodes a string to Base64 and decodes it back in Python using the base64 module. Base64 represents arbitrary bytes as plain text, which is useful when binary data must travel through text only channels such as JSON or email. Remember that Base64 is an encoding, not encryption: it provides no secrecy and grows the data by about a third. Encode for transport, not for protection.

Convert String to Binary in Python

Image
Code to convert string to binary and binary to string: def string2bits(s=''):     return [bin(ord(x))[2:].zfill(8) for x in s] def bits2string(b=None):     return ''.join([chr(int(x, 2)) for x in b]) si = input('\nenter text:\n') b = string2bits(si) so = bits2string(b) print('\nyou entered\n' + si) print('\nbinary is\n' + str(b)) print('\nstring is\n' + so) This example converts text to its binary representation and back in Python by mapping each character to its 8 bit code with ord and bin, then reversing the process with int and chr. It is a useful exercise for understanding character encoding and how text is stored as bytes underneath. For real binary handling, Python's bytes type and struct module are the practical tools.

TypeError: Can't convert 'bytes' object to str implicitly

Image
Error: Traceback (most recent call last):   File "b64-str.py", line 10, in <module>     print('\nstring is\n' + so) TypeError: Can't convert 'bytes' object to str implicitly Problem Code: so = base64.b64decode(b64) print('\nstring is\n' + so) Solution: so = base64.b64decode(b64).decode('utf-8', 'ignore') print('\nstring is\n' + so) This Python 3 error happens when you join bytes and str with the plus operator. Python 3 keeps bytes and text strictly separate, so they cannot be concatenated directly. Calling .decode() on the bytes, or .encode() on the string, converts one to the other so the types match. This is one of the most common surprises when moving code from Python 2 to Python 3.

AttributeError: module 'json' has no attribute 'dump'

Image
Problem: Today I was trying to create a json file from Python and then I got this error: "AttributeError: module 'json' has no attribute 'dump'" and I found my solution here: http://stackoverflow.com/a/13612382/4064166 Solution: So the problem was I named my file as json.py that was causing problems with json module so renaming it solved the issue. Code: import json with open("dump.json", "w") as outfile:     json.dump({'number':6650, 'strings':'lorem ipsum', 'x':'x', 'y':'y'}, outfile, sort_keys = True, indent=4, ensure_ascii=False) Screenshot:

Make file read-only in Python

Image
If you want to make file read-only: import os from stat import S_IREAD os.chmod('path/to/file.txt', S_IREAD) If you want to make file writeable: import os from stat import S_IWRITE os.chmod('path/to/file.txt', S_IWRITE) Ref. 1: http://stackoverflow.com/a/28492823/4064166 Ref. 2: http://techarttiki.blogspot.in/2008/08/read-only-windows-files-with-python.html This snippet toggles a file between read-only and writable in Python using os.chmod with stat flags. Setting S_IREAD makes it read-only; S_IWRITE makes it writable again. It is handy for protecting generated output or configuration from accidental edits. On Unix like systems the effect also depends on ownership and the existing permissions.

Create PDF from Python

Image
I was googling for a python library that I can use to write strings to a PDF file but mostly I found out programs to copy data from one PDF and write it to another PDF, another programs and solutions I found were for editing existing PDF, it was not what I was looking for. I wanted to create a PDF from scratch. After hours of searching finally I found my solution when I arrived on this page by David Fischer and it just uses one module called reportlab . Code is share via this gist on my GitHub.

TypeError: write() argument must be str, not bytes

Image
Recently I encountered this error: "TypeError: write() argument must be str, not bytes" when trying to write a PDF file from anaconda 3. The solution was to write stream in binary. Error: J:\>python pdf.py Traceback (most recent call last):   File "pdf.py", line 31, in <module>     fd.write(buf.getvalue()) TypeError: write() argument must be str, not bytes Problem code: # Write the PDF to a file with open('test.pdf', 'w' ) as fd:     fd.write(buf.getvalue()) Solution: # Write the PDF to a file with open('test.pdf', 'wb' ) as fd:     fd.write(buf.getvalue()) Ref.: http://stackoverflow.com/a/5513856 This Python error happens when binary data is written to a file opened in text mode, which is common when producing a PDF or other binary output. Opening the file in binary mode, for example with the wb flag, lets you write bytes directly and resolves it. Writing text to a file opened in binary mo...

Extract all internal and external links from a URL

Image
How it works? This is a Python script which takes complete URLs provided as command line arguments. It then parses a URL in HTML using BeautifulSoup . From the parsed webpage all the anchor hyper-references are extracted and later simple processing is done to sort them in two bins: internal and external. If link contains http or https and the URL is a part of link then it is sorted as internal link otherwise it is external link. All links starting with / or // are internal links. Other tags like javascript, mail and telephone links are ignored. All internal page jumps starting with # are ignored. This link does not provide unique list of internal/external links so if same link is present it will be counted multiple times. It will treat Top-level domain and subdomains as different URL hence it will be counted as external i.e. if you are querying for www.google.com then links with news.google.com and www.google.co.in both will be treated as different domain thus will be count...

Solve charmap codec error in Python

Image
Error: 'charmap' codec can't encode character '\u2013' in position 112: character maps to <undefined> Problem code:         for d in soup.find_all(href=re.compile(url)):             print(d) It is not an error from Python but it is because of Windows, in my case there was one character which was not getting encoded properly in command prompt (console) so I had to change the encoding to UTF-8. Solution is pretty simple just type following command. Solution: J:\>chcp 65001 Active code page: 65001 CHCP changes the active console code page. 65000 code page is encoded as UTF-7 and 65001 code page is encoded as UTF-8. ref1.: http://stackoverflow.com/a/32383309 ref2.: http://ss64.com/nt/chcp.html This Python error, the charmap codec cannot encode a character, comes from Windows defaulting to a legacy code page that cannot represent certain Unicode characters such a...

YouTube Rewind (2010 - 2016)

Image
#YouTubeRewind YouTube Rewind is a video series produced and created by YouTube and Portal A Interactive. These videos are an overview and recap of each year's viral videos, events, memes and music. Each year, the number of YouTube celebrities featured in the video, as well as the presentation of the series have increased. [Source: https://en.wikipedia.org/wiki/YouTube_Rewind ] YouTube Rewind 2010: Year in Review

Run a server from Node.js

Image
Step 1: Download and install Node.js from https://nodejs.org/en/ Step 2: Launch node from command prompt and verify it has installed correctly Step 3: Create a new file server.js as follows: var http = require('http'); var fs = require('fs'); var url = require('url'); http.createServer( function (request, response) {    var pathname = url.parse(request.url).pathname;    fs.readFile(pathname.substr(1), function (err, data)    {       if (err)       {          console.log(err);          response.writeHead(404, {'Content-Type': 'text/html'});          response.write(err.toString());       }

Create JSON from HTML in Perl

Image
Assumption: Your HTML file is not pretty-encoded i.e. all content is contained in a single line. Code: #!/usr/bin/perl use strict; use warnings; use utf8; use JSON; use autodie; open my $fh1, '<files.html' or die "Cannot open files.html: $!\n"; my $html = <$fh1>; $html =~ s/<html><head><title>JSON Viewer<\/title><\/head><body><table border="1"><thead><tr><th width="300">Keys<\/th><th width="500">Values<\/th><\/tr><\/thead><tbody>//g; $html =~ s/<\/tbody><\/table><\/body><\/html>//g; my @rows = split("<\/tr>", $html); my %data; foreach my $row (@rows) {     my @cols = split("<\/td>", $row);     my $key;         foreach my $col (@cols)     {         if (not $col =~ /<br\/>/)       ...

Create HTML from JSON in Perl

Image
#!/usr/bin/perl use strict; use warnings; use utf8; use JSON; use autodie; use HTML::TreeBuilder; my $string; {     local $/;     open my $fh1,"<files.json" or die "open failed <files.json>: $!\n";     $string = <$fh1>;     close $fh1 or die "close failed <files.json>: $!\n"; } # Parse JSON in PERL my $data = decode_json($string); # Initialize HTML my $html = "<html><head><title>JSON Viewer</title></head><body><table border=\"1\"><thead><tr><th width=\"300\">Keys</th><th width=\"500\">Values</th></tr></thead><tbody>"; # Update HTML foreach my $key (keys(%$data)) {     $html = $html."<tr><td>".$key."</td><td>";     my $values = $data->{$key};     foreach my $value (@{$values})     {  ...

Send email from Perl in Windows 10

Image
In this program I am using Gmail to send an email from Strawberry Perl on my Windows 10 system. You can install Perl from here: http://strawberryperl.com/ I installed latest Strawberry Perl 5.24.0.1 (64bit) msi and it works fine on Windows 10 Pro 1607. Next install few modules that are required to send email. Open start menu and type CPAN Client that was installed on your PC by Strawberry Perl. Open CPAN Client and type following commands to install: install Email::Send install Email::Send::Gmail install Email::Simple::Creator As we are using gmail for sending email you need to authenticate your account against gmail's smtp server with your gmail credentials and make sure you have enables POP and IMAP email forwarding on from gmail settings. Change your credentials in following program:

Top 9 Android Smartphones For 15000 Rupees

Image
1. Xiaomi Mi Max RAM: 3 GB Camera: 16/5 MP Display: 6.44" Storage: 32 GB Released: May 2016 2. Asus Zenfone 2 RAM: 4 GB Camera: 13/5 MP Display: 5.5" Storage: 16 GB Released: March 2015 3. Vivo V 3 RAM: 3 GB Camera: 13/8 MP Display: 5" Storage: 32 GB Released: April 2016

Fix update/upgrade errors in Kali Linux [Rolling Release 2016.1]

Image
Last year I published an article on this blog about fixing update and upgrade issues in Kali Linux and it helped many users and now again after a year this issue has occurred which can not be solved by previous trick hence posting a new article here. Check previous article: http://com.puter.tips/2015/03/fix-updateupgrade-errors-in-kali-linux.html This trick is better than previous one. All you need to do is check latest repository links and update it in your sources list. You can go to official Kali Linux website documents http://docs.kali.org/general-use/kali-linux-sources-list-repositories and check for the current repository. Then open /etc/apt/sources.list in your favorite editor and overwrite the file with latest repositories links you found from official website. For now you can gedit /etc/apt/sources.list and change it to as this is for latest version Kali Linux 2016.1 Rolling Release: deb http://http.kali.org/kali kali-rolling main contrib non-free # For ...

Android 7.0 Nougat list of supported SOC

Image
Supported processors are as below: Qualcomm Snapdragon 415 430 435 615 616 617 625 650 652 805 808 810 820 821 Samsung Exynos ARMv8 5433 7420 7870 8890 Huawei Kirin 950, 955 MediaTek MT with Mali 6732 6722 6755 6797 MediaTek MT with PowerVR 6595M 6595T 6595M 6795 8135 8173 8176 679x Intel Atom Z 3460 3480 3530 3560 3570 3580 Rockchip RK 3368 Nvidia Tegra K1, X1 Nvidia GeForce 600 700 800 900 1000 Unsupported processors are as below: Qualcomm Snapdragon 200 208 210 212 400 410 412 600 800 801 Samsung Exynos 5250 5260 5410 5420 5422 5430 5800 7580 Mediatek MT 6735, 6753 Huawei Kirin 920 925 930 935 Learn more about Android 7.0 Nougat here: https://www.android.com/versions/nougat-7-0/

Learn C and C++

Image
C and C++: The major difference between C and C++ is that C is a procedural programming language and does not support classes and objects, while C++ is a combination of both procedural and object oriented programming language; therefore C++ can be called a hybrid language. C: C was developed by Dennis Ritchie between 1969 and 1973 at AT&T Bell Labs. When compared to C++, C is a subset of C++. C supports procedural programming paradigm for code development. In C (because it is a procedural programming language), data and functions are separate and free entities.

Full (MEAN) Stack Developer Tutorials

Image
Full Stack Development: The term full-stack means developers who are comfortable working with both back-end and front-end technologies. To be more specific, it means that the developer can work with databases, PHP, HTML, CSS, JavaScript and everything in between, also, venturing as far as converting Photoshop designs to front-end code. (Ref.: https://www.sitepoint.com/full-stack-developer/ ) MEAN Stack: MEAN is a framework for an easy starting point with MongoDB, Node.js, Express, and AngularJS based applications. It is designed to give you a quick and organized way to start developing MEAN based web apps with useful modules like Mongoose and Passport pre-bundled and configured. We mainly try to take care of the connection points between existing popular frameworks and solve common integration problems. (Ref.: http://learn.mean.io/ ) Online Certification Courses (MOOCs) and Tutorials: https://www.udacity.com/course/full-stack-web-developer-nanodegree--nd004 h...

Fix _nsis.py installation error in Anaconda

Image
What is Anaconda? Anaconda is the leading open data science platform powered by Python. The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for data science. Ref.: https://www.continuum.io/downloads The error reads as below: Traceback (most recent call last): File "C:\Anaconda2\Lib\_nsis.py", line 164, in <module> main() File "C:\Anaconda2\Lib\_nsis.py", line 150, in main   mk_menus(remove=False) File "C:\Anaconda2\Lib\_nsis.py", line 94, in mk_menus   err("Traceback:\n%s\n" % traceback.format_exc(20)) IOError: [Errno 9] Bad file descriptor

Active Directory login in GitBlit

Image
What is Git? Git is a version control system that is used for software development and other version control tasks. As a distributed revision control system it is aimed at speed, data integrity, and support for distributed, non-linear workflows. Git was created by Linus Torvalds in 2005 for the development of the Linux kernel, with other kernel developers contributing to its initial development. Read more about Git: https://en.wikipedia.org/wiki/Git_(software) What is GitBlit? Gitblit is an open-source, pure Java stack for managing, viewing, and serving Git repositories. It's designed primarily as a tool for small workgroups who want to host centralized repositories. Read more about GitBlit: http://gitblit.com/

Host shiny server in Ubuntu 14.04 LTS

Image
I have installed shiny server on Ubuntu 14.04 LTS running under a VM. This post is a collection of commands that I had used for setup. Note that if you want to use shiny server on LAN then use Bridged Network Adapter. On VM you can access the shiny server via localhost:3838 and on remote machines, you can use IP-address:3838 to open it. You can host your shiny app in the /srv/shiny-server/ directory. You can go to /etc/shiny-server/shiny-server.conf to change default configurations like you can change port with listen <port number>; in server {} module. $ sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/ubuntu trusty/" >> /etc/apt/sources.list' $ sudo gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9 $ sudo gpg -a --export E084DAB9 | sudo apt-key add - $ sudo apt-get update $ sudo apt-get -y install r-base $ sudo su - -c "R -e \"install.packages(c('shiny','shinyjs','shinydashboard','...

Generate word cloud in R

Image
TEXT (.txt) This working script was tested on R 3.2.5 . Code is adapted from https://github.com/gimoya/theBioBucket-Archives/blob/master/R/txtmining_pdf.R . It reads a text file, processes it to remove unnecessary words and plots it. Code: library(tm) library(wordcloud) library(Rstem) filetxt <- "C:\\Users\\310211146\\Documents\\Other\\May_Report.txt" txt <- readLines(filetxt) txt <- tolower(txt) txt <- removeWords(txt, c("\\f", stopwords())) corpus <- Corpus(VectorSource(txt)) corpus <- tm_map(corpus, removePunctuation) tdm <- TermDocumentMatrix(corpus) m <- as.matrix(tdm) d <- data.frame(freq = sort(rowSums(m), decreasing = TRUE)) d$stem <- wordStem(row.names(d), language = "english") d$word <- row.names(d) d <- d[nchar(row.names(d)) < 20,] agg_freq <- aggregate(freq ~ stem, data = d, sum) agg_word <- aggregate(word ~ stem, data = d, function(x)   x[1]) d <- cbind(freq...

Learn R with 5 Video Tutorials

Image
If you are new to R or wants to enhance your basic skills in R and wondering where to get started provided there is lots of free material available online then you have come to right place. I have presented 5 video tutorials that is more than enough to get you started with R and will push you in direction of excelling R. 1. What is R and How to install it? 2. What are data types and variables in R?

R: Count occurrences in vector

Image
If you are having a vector and you want to count frequency of individual member (in other words, occurrences of each element) then you can do as following: use hist() function: x <- c(1,3,2,5,4,4,2,2,4,2,4,2,4,6,4,5,2)   x [1] 1 3 2 5 4 4 2 2 4 2 4 2 4 6 4 5 2   hist(x) or use summary() function: x <- factor(x)   x [1] 1 3 2 5 4 4 2 2 4 2 4 2 4 6 4 5 2 Levels: 1 2 3 4 5 6   summary(x) 1 2 3 4 5 6 1 6 1 6 2 1   convert vector into factor and then call summary function. OR if you are working with characters then hist() won't be useful as it expects numeric elements so use summary() as shown below:   x <- c("devharsh","trivedi","R","C#","ASP.NET","R","C#","devharsh","R")   x <- factor(x)   summary(x) ASP.NET C# devharsh R trivedi 1 2 2 3 1

5.5.1 Authentication Error in Gmail

Image
Code for sending email using gmail in C# : using System.Net.Mail; using System.Net; MailMessage mail = new MailMessage(); mail.To.Add("dummy@test.com"); mail.From = new MailAddress("dummy-sender@gmail.com"); mail.Subject = "subject"; mail.SubjectEncoding = System.Text.Encoding.UTF8; mail.Body = "foo foo bar"; mail.Priority = MailPriority.High; SmtpClient client = new SmtpClient(); client.UseDefaultCredentials = false; client.Credentials = new NetworkCredential("dummy-sender@gmail.com", "gmail-password"); client.Port = 587; client.Host = "smtp.gmail.com"; client.EnableSsl = true; client.Send(mail); Error: "The SMTP server requires a secure connection or the client was not authenticated. The server response was: 5.5.1 Authentication Required" Solution: Login to your gmail account from which you are trying to send email and then go to https://www.google.com/settings/security/lesss...

Fix _CRT_SECURE_NO_WARNINGS in VC++

Image
Error: To disable deprecation, use _CRT_SECURE_NO_WARNINGS. Error    1    error C4996: 'sscanf': This function or variable may be unsafe. Consider using sscanf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. Error    1    error C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. Solution: Go to Project -> Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions and add _CRT_SECURE_NO_WARNINGS in the list. This Visual C++ warning, C4996, flags standard C functions such as sscanf and fopen as unsafe because they do not bound their buffers, and MSVC suggests the _s secure variants instead. You can switch to the suggested secure functions, or define _CRT_SECURE_NO_WARNINGS to silence the warning once you have rev...

cannot coerce type 'externalptr' to vector of type 'character'

Image
Error: Error in as.vector(x, "character") :   cannot coerce type 'externalptr' to vector of type 'character' Problem Code: con <- dbConnect(RMySQL::MySQL(), dbname = "test") sql <-     paste0(       "INSERT INTO `devharsh` (`un`, `pw`, `bl`) VALUES ('", TextUN ,"', '", TextPW ,"', '",0,"')"     )   dbGetQuery(con, sql) Solution: sql <-     paste0(       "INSERT INTO `devharsh` (`un`, `pw`, `bl`) VALUES ('", TextUN$getText() ,"', '", TextPW$getText() ,"', '",0,"')"     ) Note: It was giving error because it was not getting text from textbox that we should get by getText() function. This R error means a value that is not a plain character vector was passed where a string was expected. Here a database connection handle, an external pointer, was used where text was needed. The fix is to pass th...

CPU and Memory Usage in C#

Image
using System; using System.Management; namespace WMISample {      public class MyWMIQuery     {          public static void Main ()         {              try             {                  ManagementObjectSearcher searcher =                      new ManagementObjectSearcher ( "root\\CIMV2" ,                                            ...

Compare two files with PSPad

Image
You can go to http://www.pspad.com/en/download.php and download PSPad text editor. To get the difference between two text files open PSPad editor. Then go to Tools menu and select Text Differences then select first option that is Text Diff with This File...

Input string was not in a correct format in ASP.NET/C#

Image
ERROR:   Server Error in '/' Application. Input string was not in a correct format. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.FormatException: Input string was not in a correct format. Source Error: Line 31:         int ppi = Convert.ToInt32(Text1.Text); SOLUTION: I tried to change int ppi = Convert.ToInt32(Text1.Text); statement to int ppi = int.Parse(Text1.Text); and int ppi = Int32.Parse(Text1.Text); and int ppi = Int16.Parse(Text1.Text); but it didn't work. The reason was I was using a float value and parsing into an integer one, so I change the erroneous statement to the following, and it worked: float ppi =  float.Parse(Text1.Text);

Data analysis not available in Excel

Image
If you are using MS Excel version older than 2016 and if Data analysis option is not available in Analysis section under Data tab then follow the instructions below. Data analysis option is not available. Go to File menu > Options > Add-Ins > Select "Excel Add-ins" in Manage drop-down list that is located at the end of the window and hit go. Then select Analysis ToolPak and click OK it will add Data Analysis option under Analysis group in Data tab. This is how it looks like after enabling the add-in.