Journey of Analytics

Deep dive into data analysis tools, theory and projects

Page 2 of 7

How to Become a Data Scientist

This question and its variations are the most searched topics on Google. As a practicing datascience professional, and manager to boot, dozens of people ask me this question every week.

This post is my honest and detailed answer.

Step 1 – Coding & ML skills

  • You need to master programming in either R or Python. If you don’t know which to pick, pick R, or toss a coin. [Or listen to me, and pick R – programming as it is used at Top Firms like NASDAQ, JPMorgan, and many more..] Also, when I say master, you need to know more than writing a simple calculator or “Hello World” function. You should be able to perform complex data wrangling, pull data from databases, write custom functions and apply algorithms, even if someone wakes you up at midnight.
  • By ML, I mean the logic behind machine learning algorithms. When presented with a problem, you should be able to identify which algorithm to apply and write the code snippet to do this.
  • Resources – Coursera, Udacity, Udemy. There are countless others, but these 3 are my favorites. Personal recommendation, basic R from Coursera (JHU) and Machine learning fundamentals from Kirill’s course on Udemy.

Step 2 – Build your portfolio.

  • Recruiters and hiring managers don’t know you exist, and having an online portfolio is the best way to attract their attention. Also, once employers do come calling, they will want to evaluate your technical expertise, so a portfolio helps.
  • The best way to showcase your value to potential employers is to establish your brand via projects on Github, LinkedIn and your website.
  • If you do not have your own website, create one for free using wordpress or Wix.
  • Stumped on what to post in your project portfolio?
  • Step1 – Start by looking in the kernels portion on the site www.kaggle.com there are tons of folks who have leveraged free datasets to create interesting visualizations. Also enroll in any active competitions and navigate to the discussion forums. You will find very generous folks who have posted starter scripts and detailed exploratory analysis. Fork the script and try to replicate the solution. My personal recommendation would be to begin with titanic contest or the housing prices set. My professional website journeyofanalytics also houses some interesting project tutorials, if you want to take a look.
  • Step 2 – pick a similar datasets from kaggle or any other open source site, and apply the code to the new datasets. Bingo, a totally new project and ample practice for you.
  • Step3 – Work your way up to image recognition and text processing.

Step 3 – Apply for jobs strategically.

  • Please don’t randomly apply to every single datascience job in the country. Be strategic using LinkedIn to reach out to hiring managers. Remember, its better to hear “NO” directly from the hiring manager than to apply online and wait in eternity.
  • Competition is getting fierce, so be methodical. Books like “Data Science Jobs” will help you pinpoint the best jobs in your target city, and also connect with hiring managers for jobs that are not posted anywhere else.
  • Yes, I wrote the book listed above – this is the book I wished I had when I started in this field! Unlike other books on the market with random generalizations, this book is written specifically for jobseekers in the datascience field. Plus, I’ve successfully helped a dozen folks land lucrative jobs (data analyst/data scientist roles) using the strategies outlined in this book. This book will help you cut your datascience job search time in half!
  • Upwork is a fabulous site to get gigs to tide you until you get hired full-time. It is also a fabulous way of being unique and standing out to potential employers! As a recruiter once told me, “it is easier to hire someone who already has a job, than to evaluate someone who doesn’t!”
  • If your first job is not at your dream job, do not despair. Earn and learn, every company, big or small, will teach you valuable skills that will help you get better and snag your ideal role next year. I do recommend staying at roles for at least 12 months, before switching, otherwise you won’t have anything impactful to discuss in the next interview.

Step 4 – Continuous learning.

  • Even if you’ve landed the “data scientist” job you always wanted, you cannot afford to rest on your laurels. Keep your skills current by attending online classes, conferences and reading up on tech changes.
    Udemy, again is my go to resource to stay abreast of technical skills.
  • Network with others to know how roles are changing, and what skills are valuable.

Finally, being in this filed is a rewarding experience, and also quite lucrative. However, no one can get to the top without putting in sufficient effort and time. So, master the basics and apply your skills, you will definitely meet with success.

If you are looking to establish a career in datascience, then don’t forget to take a look at my book – “Data Science Jobs‘ now available on Amazon.

Data Science Job in 90 days – Book Review

Are you an R-programmer or Datascience enthusiast looking for a break in the datascience field? If so, my latest book “Data Science Jobs – land a lucrative job in 90 days” will help you find one quickly.

Imagine reducing your job hunt time by 2 weeks, or even 4 weeks? The strategies in this book are designed to do just that. [ Amazon book link here.]

As an analytics manager I get countless requests for job search advice, resume feedback and brilliant students who are somehow unable to find a job in this exciting field. There are tons of books on the internet on how to learn the skills to become a data scientist/ data analyst, but none to prepare folks for the frustrating job search.

I repeat the same advice to all these requests and am delighted to say that a dozen people were successfully able to land their dream roles with companies like LinkedIn, Walmart Labs, Comcast and others. I decided to publish the book so others can also benefit from the same advice.

Target Reader Audience

  • Students with solid knowledge of programming in R or Python looking to find a role as a data analyst/scientist or BI developer. A background in computer science or math will help, but not necessary.
  • International student on F-1/OPT visa looking for employment after a graduate degree in analytics.
  • Employed professionals looking to pivot their career, or seeking better pay/manager/location.

Book Summary

The book aims to provide you with creative techniques to get your resume directly in the hands of hiring managers, instead of relying hopelessly on online application systems that rarely produce a response. Don’t be fooled by the length of the book – it is deliberately kept short so that jobseekers can read through quicky and apply these principles in their job search.

The book chapters provide detailed guidelines on these broad themes:

  • Personal Branding – Create an online profile that helps you bubble up when hiring managers look for candidates. Make the jobs come to you! Tips to tweak your resume to achieve the same. For project inspiration, look at learning communities like R-bloggers, Kaggle, etc.
  • LinkedIn – Secret ways to leverage LinkedIn to engage hiring managers. Do NOT simply accept connections or indiscriminately apply to every open job. How to use LinkedIn to improve personal SEO!
  • Strategic Networking – How to actively reach out to the decision-makers who can hire you!
  • Niche sites – Hiring managers understand that the best venue to hire talent are the datascience communities where folks go to learn. The book lists these niches job boards on sites like R-bloggers, Kaggle.com and many others.
  • Upwork – despite popular opinion (about the site’s ineffectiveness), this site is a quick way to earn money and position yourself for your dream role.
  • And many more…

In conclusion, this book is a condensed guide with practical strategies to make the job search process less stressful, and help readers quickly get hired. So get the ebook on Amazon, and get started on a lucrative career!

Top 5 Secret Tips to make the most of your online MBA

Online and executive MBAs are becoming more common and widely respected as more professionals balk at the hefty price tag of regular degrees, and the opportunity cost of leaving a cushy job.

With many employers also supporting tuition reimbursements, and colleges becoming better supporting their online students, this is a trend that shows no signs of slowing down.

However, it can get lonely and many online students do not fully leverage the opportunities of an online MBA and often feel a little disillusioned at not being able to attract the networking groups that traditional in-person programs offer. All articles on how to make the most of an MBA seem catered towards those who have the luxury of doing their courses full-time.

So here are top 5 tips to help you make the most of your online MBA. If you are taking any other online degree, then fear not, these techniques still apply.

Why are they “secret” ? Because no one ever tells you, because “everyone” is supposed to know. But students never do, until its too late to do anything about it… So without further ado, here are the tips to maximize your online degree or MBA program.

  1. Time Management.
  2. Go beyond the minimum required coursework.
  3. Leverage LinkedIn to forge better connections.
  4. Tell your manager.
  5. Showcase your skills.

What makes me qualified to talk about online degree programs? I am 80% complete through my online MBA while holding a full-time job, and maintaining ~3.6 GPA. Received A/B for almost all my classes. Despite not taking any classes last summer, I am on track to complete my MBA in less than 20 months, since I started in Aug 2017. A lot of folks have asked me this question, so I figured a post might be helpful to others who don’t know me personally, and looking for the same info.

1. Time Management

If you are working full-time and/or have a family, then one of the first things you will notice is that you are pressed for time. BIG TIME. Do not worry, this is a skill you need to master as an MBA, and future manager. You will have to juggle and excel at working on multiple (and often conflicting) priorities, so it is best you learn this well now.

To me personally, studying was easy, finding time to do it was incredibly challenging. So how can you cope?

  1. First make a list of your daily routine. Include office hours, commute time, travel plans, kids’ activities, etc.
  2. Now assign time for studies. You may find that you need to delegate some stuff to your partner, older kids, or give up some items altogether. Or you may realize that taking 2 courses per semester is a stretch.
  3. Get creative. If you commute by train, can you read on the train. If you commute by car, there are software programs that can “read” textbook content as audio files, so you can listen in your car. Do you need to block 2 hours in the weekend when someone else can look after the kids, so you can head to the library to study in silence?
  4. Some employers are a little accommodating too. For example, one friend told me how his manager allowed him to book a huddle room for 2 hours per week, so he could sit and watch his class videos.
  5. I used to buy used bad-condition textbooks so I could literally tear off 10 pages of a chapter, and carry them around everywhere. So I could read while waiting in doctor’s clinics, connecting trains, once even in a serpentine queue in the post office. I had to bind these books before selling them back, others I donated. The “A” grades I received were worth the effort.
  6. Guard your time ruthlessly. Once you’ve found time for studying, do not use it for personal appointments, cleaning the house, getting a facial or any of the 101 things that we all have in our to-do list. Study time is for study ONLY.
  7. Create a calendar with deadlines for quizzes, discussions, papers due, etc. Keep additional deadlines 1 or 2 days before the due date, so you have buffer for completing them.
  8. Know the scores needed to make the grade. If your employer is paying partially or fully, know what is the cutoff. Some employers want a B+ or above, others will settle for a C or above. However, colleges also have a mandatory threshold for grad students, and you can go on probation or pay for recovery grade classes, if you receive less than Bs in too many classes.
  9. If your class allows, try to complete as much work in the first 2-3 weeks, so you have leeway to lose scores in the midterms and finals, which are typically harder. For my first class, I realized A-grade was scored at 95%! What????!! I lost 20 points out of 1000 in the first quiz itself, so I knew within 2 weeks that an A for that class was not possible. However, I did make it to a B (85%-95%) and thanks to an optional assignment, just scraped into the A-grade. Being familiar with your syllabi is crucial to get this done.

2. Go beyond minimum coursework.

This may sound ironical, given that finding time for regular work itself is challenging. However, you can still do it by being smart about your work and your time.

  1. For online discussions, if you have to post 2 peer responses, post 3. Most colleges have a smartphone app, so you can easily do this in small chunks of time (lunchbreak or waiting for a boring meeting to start).
  2. Look for responses that are completely counter to your argument, so you can really view the topic from a fresh perspective.
  3. Do try to work on at least a couple self-assessment Qs from the back of the chapters, and look it up on Google Scholar or regular Google. If in doubt, ask your teaching assistant or professor via email. Most of them will be delighted to help you understand.

3. Leverage LinkedIn.

This is no-brainer, but I despair at the people who still fail to follow it.

  1. Update your LinkedIn profile with the degree program you are pursuing. If you don’t have a LinkedIn profile, then for heavens sake create one.
  2. All major colleges have student groups, and alumni groups. Add yourself to both. Then contribute meaningful conversations, without spamming folks.
  3. A lot of professors, assistant professors and teaching assistants are also on LinkedIn. So connect with them, by adding a note about the class you took with them. Professors rarely refuse.
  4. Look for classmates and alumni who work in your industry/ company/ area, and invite them to connect. Make sure to add a note about being fellow alumni. Perhaps you could even ask them which courses they liked or found most challenging. Most folks are very generous on LinkedIn.
  5. Add your LinkedIn url (customize them please) to your signature, and add the link to all the emails and conversations you have in school. This includes introductions (every class will have this), posts. Don’t add to peer responses, if you feel hesitant, but definitely include in emails and group project conversations, etc. Online learning platforms don’t allow this very easily, so I had it pasted as a note on my desktop and manually added it to every introductory conversation. If this sounds creepy and self-pushing to you, let me add that I have completed 8 classes with 3.6 GPA and till date no one has ever called me out on it. The 50+ classmates who connected with me on their own, was totally worth it though.
  6. For introductions, read through what others are doing and saying. Even if they are not in your location, do say hi, and request to connect. You never know who will be able to help out whom. For example, I was able to connect 7 of my classmates from different courses (3 pairs basically) because they lived in same area, and 2 worked for the same company and location, without knowing each other. Many others have helped me understand concepts and with homework when I was struggling, and one motivated me via daily LinkedIn messages when I was feeling completely overwhelmed.
  7. Aim to connect with at least 5 people from each class. Plus, like all LinkedIn connections stay connected beyond the class. Send them hello for New year or Thanksgiving and congratulate them on role changes, birthdays and so on.
  8. If you work on folks for group projects, then do send them a request to connect beyond the class. These folks are at least a good source for skill endorsements and recommendations.

4. Tell your manager.

  1. Irrespective of whether your employer is reimbursing your course or not, do tell them about the course. Tell them your hopes and expectations from the course. Most of us do hope for a promotion, and salary hike, so having your manager in the loop helps.
  2. Do not tell your manager about your MBA as a threat; and definitely do not use it as a hostile negotiation tactic. Instead tell them that you are looking to improve your skills and how you hope your new skills will help you increase your value in the team. You may be surprised how happy your manager is with your proactive nature, and may even offer you additional projects to help you apply your skills.
  3. If possible, tell your manager before or during the application process itself, as they may be able to tell you about partner universities, or help you connect with others who have taken similar courses. My manager (at Nasdaq) did tell me about a great subsidized program, at a college right across the street, although I finally ended up joining a totally different program. However, he did introduce me to 2 amazing colleagues in unrelated departments who were also pursuing executive MBAs (diff university). Their tips helped me navigate my program more efficiently.
  4. Keep your manager updated about courses, so he/she knows how you are doing and all the fantastic skills you are picking up! You don’t really need to tell them exact grades if you don’t want too, but if you got good ones, TELL. Believe me, it will come up in conversations with his peers and seniors, and you will be glad to have a positive note worth sharing.
  5. Obviously, your work should be priority over the degree, since your pay depends on it. But should a problem arise, your manager at least knows you have other deadlines, and may help you when you are crunched during an important final. Don’t make it a habit of it, or your manager may question if you really should be pursuing the course, or if you really have the ability to take on additional responsibilities.

5. Showcase your skills.

This is crucial, yet I am amazed how many people never bother doing it.

  1. LinkedIn allows you to add courses, so add ones that will probably work as keywords. This is apart from your educational qualifications. For example, my profile lists data analysis, financial accounting for managers, and strategy management. As a strategy/risk analyst, these are quite relevant. This is aside from the MBA that is listed on my education section.
  2. You will be doing projects for courses, so add a summary in the projects section. If your classmates are on LinkedIn, tag them as well. This helps to boost your profile.
  3. This article from Udacity graduate Nirupama, has excellent tips on how to make the most of project-based courses. It was written for MOOCs, but translates very well for courses from online degree programs.
  4. Add skills. LinkedIn allows you to allow up to 50 skills, so make sure yours are the most relevant and important, from the perspective of your current role, and the role you want to get into. Plus, you can ask your classmates to endorse you for these skills, as they know firsthand that you worked to learn them during the course. Remember to return the favor to them as well. I normally endorse the top 5 skills, and ask them if there are specific ones that they want to bubble up to the top. (LinkedIn has some sort of algorithm, so the ones with the most votes, generally show up on top. You can re-order manually)
  5. Ask classmates to add recommendations.
  6. Ask professors/ teaching assistants to recommend you, if you did particularly well in class. I did have one professor who said he doesn’t recommend people on LinkedIn, since he has thousands of students, but he did send me a very nice email note, and agreed to serve as a reference source should I need one. Not ideal, but quite helpful.

Those were the tips that I found most helpful to ace my MBA, and hopefully should make the journey easier for you as well. So study well, and enjoy your program!

DataScience Professionals : India vs US | Men vs Women

Introduction

This is an analysis of the Kaggle 2018 survey dataset. In my analysis I am trying to understand the similarities and differences between men and women users from US and India, since these are the two biggest segments of the respondent population. The number of respondents who chose something other than Male/Female is quite low, so I excluded that subset as well.

The complete code is available as a kernel on the Kaggle website. If you like this post, do login and upvote! 🙂  This post is a slightly truncated version of the Kernel available on Kaggle.

You can also use the link to go to the dataset and perform your own explorations. Please do feel free to use my code as a starter script.

 

Kaggle users - India vs US

Kaggle users – India vs US

Couple of disclaimers:

NOT intending to say one country is better than the other. Instead just trying to explore the profiles based on what this specific dataset shows.
It is very much possible that there is a response bias and that the differences are due to the nature of the people who are on the Kaggle site, and who answered the survey.
With that out of the way, let us get started. If you like the analysis, please feel to fork the script and extend it further. And do not forget to Upvote! 🙂

Analysis

Some questions that the analysis tries to answer are given below:
a. What is the respondent demographic profile for users from the 2 countries – men vs women, age bucket?
b. What is their educational background and major?
c. What are the job roles and coding experience?
d. What is the most popular language of use?
e. What is the programming language people recommend for an aspiring data scientist?

I deliberately did not compare salary because:
a. 16% of the population did not answer and 20% chose “do not wish to disclose”.
b. the lowest bracket is 0-10k USD, so the max limit of 10k translates to about INR 7,00,000 (7 lakhs) which is quite high. A software engineer, entering the IT industry probably makes around 4-5 lakhs per annum, and they earn much more than others in India. So comparing against US salaries feels like comparing apples to oranges. [Assuming an exchange rate of 1 USD = 70 INR].

Calculations / Data Wrangling:

  1. I’ve aggregated the age buckets into lesser number of segments, because the number of respondents tapers off in the higher age groups. They are quite self-explanatory, as you will see from the ifesle clause below:
  2. Similarly, cleaned up the special characters in the educational qualifications. Also added a tag to the empty values in the following variables – jobrole (Q6), exp_group (Q8), proj(Q40), years in coding (Q24), major (Q5).
  3. I also created some frequency using the sqldf() function. You could use the summarise() from the dplyr package. It really is a matter of choice.

Observations

Gender composition:

As seen in the chart below, many more males (~80%) responded to the survey than women (~20%).
Among women, almost 2/3rd are from US, and only ~38% from India.
The men are almost split 50/50 among US and India.

 

Age composition:

There is a definite trend showing that the Indian respondents are quite young, with both men and women showing >54% in the youngest ge bucket (18-24), and another ~28% falling in the (25-29) category. So almost 82% of the population is under 30 years of age.
Among US respondents, the women seem a bit more younger, with 68% under 30 years, compared to ~57% men of women. However, both men and women had a larger segment in the 55+ category (~20% for women, and 25% for men. Compare it with Indians, where the 55+ group is barely 12%.

 

Educational background:

Overall, this is an educated lot, and most had a bachelors degree or more.
US women were the most educated of the lot, with a whopping 55% with masters degrees and 16% with doctorates.
Among Indians, women had higher levels of education – 10% with Ph.D, 43% masters degree, compared with men where ~34% had a masters degree and only 4% had a doctorate.
Among US men, ~47% had a masters degree, and 19% had doctorates.
This is interesting because Indians are younger compared to US respondents, so many more Indians seem to be pursuing advanced degrees.

Undergrad major:

Among Indians, the majority of respondents added Computer Science as their major.
Maybe because :
(a) Indians have to declare a major when they join, and the choice of majors is not as wide as in the US. ,

  1. Parents tend to force kids towards majors which are known to translate into a decent paying job, which is engineering or medicine.
  2. A case of response bias? The survey came from Kaggle, so not sure if non-coding majors would have even bothered to respond.Among US respondents, the major is also computer science, but followed by maths & stats for women.
    For men, the second category was a tie between non-compsci Engg , followed by maths&stats.

 

Job Roles:

Among Indians, the biggest segment are predominantly students (30%). Among Indian men, the second category is software engineer.
Among US women, the biggest category was also “student” but followed quite closely by “data scientist”. Among US men , the biggest category was “data scientist” followed by “student”.
Note, “other” category is something we created now, so not considering those. They are not the biggest category for any sub-group anyway.
CEOs, not surprisingly are male, 45+ years from the US, with a masters degree.

 

Coding Experience:

Among Indians, most answered <1 year of coding experience , which correlates well with the fact that most of them are under 30, with a huge population of students.
Among US respondents, the split is even between 1-2 years of coding and 3-5 years of coding.
Men seem to have a bit more coding experience than women, again explained by the fact that women were slightly younger overall, compared to US men.

 

Most popular programming language:

Python is the most popular language, discounting the number of people who did not answer. However, among US women, R is also popular (16% favoring it).

I found this quite interesting because I’ve always used R at work, at multiple big-name employers. (Nasdaq, Td bank, etc.) Plus, a lot of companies that used SAS seem to have found it easier to move code to R. Again this is personal opinion.
Maybe it is also because many colleges teach Python as a starting programming language?

 

Conclusions:

  1. Overall, Indians tended to be younger with more people pursuing masters degrees.
  2. US respondents tended to older with stronger coding experience, and many more are practicing data scientists.
    This seems like a great opportunity for Kaggle, if they could match the Indian students with the US data scientists, in a sort of mentor-matching service. 🙂

Automated Email Reports with R

R is an amazing tool to perform advanced statistical analysis and create stunning visualizations. However, data scientists and analytics practitioners do not work in silos, so these analysis have to be copied and emailed to senior managers and partners teams. Cut-copy-paste sounds great, but if it  is a daily or periodic task, it is more useful to automate the reports. So in this blogpost, we are going to learn how to do exactly that.

The R-code uses specific library packages to do this:

  • RDCOMClient – to connect to Outlook and send emails. In most offices, Outlook is still the defacto email client, so this is fine. However, if you are using Slack or something different it may not work.
  • r2excel – To create an excel output file.

The screenshot below shows the final email view:

email screenshot

email screenshot

As seen in the screenshot, the email contains the following:

  • Custom subject with current date
  • Embedded image
  • Attachments – 1 Excel and 1 pdf report

Code Explanation:

The code and supporting input files are available here, under the Projects page under Report automation. The code has 4 parts:

  • Prepare the work space.
  • Pull the data from source.
  • Cleaning and calculations
  • Create pdf.
  • Create Excel file.
  • Send email.

The workflow should look something like this. For really simple reports, where you do not need much formatting, you can do everything within the first script itself.

Prepare the work space

I always set the relative paths and working directories at the very beginning, so it is easier to change paths later. You can replace the link with a shared network drive path as well.

Load library packages and custom functions. My code uses the r2excel package which is not directly available as an R-cran package. So you need to install using devtools using the code below.

It is possible to do something similar using the “xlsx” package, but r2excel is easier.

Some other notes:

  • you need the first 2 lines of code only for the first time you installation. From the second time onwards, you only need to load the library.
  • r2excel seems to work only with 64-bit installations of R and Rstudio.
  • you do need Java installed on your computer. If you see an error about java namespace, then check the path variables. There is a very useful thread on Stackoverflow, so take a look.
  • As always, if you see errors Google it and use the Stack Overflow conversations. In 99% of cases, you will find an answer.

Pull the data from source

This is where we connect to an Excel CSV (or text) file. In practice, most people connect to a database of some kind. The R-script I am using connects to a .csv file, but I have added the code to a connect to a SQL database.

That code snippet is commented out, so feel free to substitute your own sql database links. The code will also work for Amazon EC2 cluster.

Some points to keep in mind:

  • If you are using sqlquery() then please note that if your query has an error then R sadly shows only a standard error message. So test your query on SQL server to ensure that you are not missing anything.
  • Some queries do take a long time, if you are pulling from a huge dataset. Also the time taken will be longer in R compared to SQL server direct connection. Using the  Sys.time() command before and after the query is helpful to know how long the query took to complete.
  • If you are only planning to pull the data randomly, it may make sense to pull from SQL server and store locally. Use the fread() function to read those files.
  • If you are using R desktop instead of R-server, the amount of data you can pull may be limited to what your system configuration.
  • ALWAYS optimize your query. Even if you have unlimited memory and computation power, only pull the data you absolutely need. Otherwise you end up unnecessarily sorting through irrelevant data.

Cleaning and calculations

For the current data, there are no NAs, so we don’t need to account for those. However, the read.csv() command creates factors, which I personally do not like, as they sometimes cause issues while merging.

Some of the column names have “.” where R converted the space in the names. So we will manually replace those with an underscore using the gsub() function.

We will also rank the apps based on categories of interest, namely:

  • Most Popular Apps – by number of Reviews
  • Most Popular Apps – by number Downloads and Reviews
  • Most Popular Categories – Paid Apps only
  • Most popular apps with 1 billion installations.

Create pdf

We are going to use the pdf() function to paste all graphs to a pdf document. Basically what this function does is write the graphs to a file rather than show on the console. So the only thing to remember is that if you are testing graphs or make an incorrect graph, everything will get posted to the pdf until you hit the “dev.off()” function. Sometimes if the graph throws an error you may end up with a blank page, or worse, with a corrupt file that cannot be opened.

Currently, the code I am only printing 2 simple graphs using ggplot() and barplot() functions, but you can include many other plots as well.

Create Excel file.

The Excel is created in the sequence below:

  • Specify the filename and create an object of type .xlsx This will create an empty Excel placeholder. It is only complete when you save the Workbook using the saveWorkbook() at the end of the section.
  • Use the sheets() to create different worksheets within the Excel.
  • The  xlsx.addHeader() adds a bold Header to each sheet which will help readers understand the content on the page. The r2excel package has other functions to add more informative text in smaller (non-header) font as well, if you need to give some context to readers. Obviously, this is optional if you don’t want to add them.
  • xlsx.addTable() – this is the crucial function that adds the content to Excel, the main “meat” of what you need to show.
  • saveWorkbook() – this function will save the Excel to the folder.
  • xlsx.openFile() – this function opens the file so you can view contents. I typically have the script running on automated mode, so when the Excel opens I am notified that the script completed.

Send email

The email is sent using the following functions:

  • OutApp() – creates an Outlook object. As I mentioned earlier, you do need Outlook and need to be signed in for this to work. I use Outlook for work and at home, so I have not explored options for Slack or other email clients.
  • outmail[[“To”]] – specify the people in the “to” field. You could also read email addresses from a file and pass the values here.
  • outmail[[“cc’]] – similar concept, for the cc field.
  • outmail[[“Subject”]] – I have used the paste0() function to add the current date to the subject, so recipients know it is the latest report.
  • outMail[[“HTMLBody”]] – I used the HTML body so that I can embed the image. If you don’t know HTML programming, no worries! The code is pretty intuitive, you should be able to follow what I’ve done. The image basically is an attachment which the HTML code is forcing to be viewed within the body of the email. If you are sending the email to people outside the organization, they may see a small box instead of the image with a cross on the top left (or right) of the box. Usually, when you hover your mouse near box and right click, it will ask them to download images. You may have seen similar messages in gmail, along with a link to “show images” or ‘always show images from this sender’. You obviously cannot control what the recipient selects, but testing by sending to yourself first helps smoothing out potential aesthetic issues.
  • outMail[[“Attachments”]] – function to add attachments.
  • outMail$Send() – until you run this command, the mail will not be send. If you are using this in office, you may get a popup asking you to do one of the following. Most  of these will generally go away after the first use, but if they don’t, please look up the issue on StackOverflow or contact your IT support for firewall and other security settings.
    • popup to hit “send”
    • popup asking you to “classify” the attachments (internal / public/ confidential) Select as appropriate. For me, this selection is usually  “internal”
    • popup asking you to accept “trust” settings
    • popup blocker notifying you to allow backend app to access Outlook.

That is it – and you are done! You have successfully learned how to send an automated email via R.

« Older posts Newer posts »
Facebook
LinkedIn