Protect Your Meeting’s Productive Time

by Xavier Comments: 0

Time flies.

Time is of the essence.

I don’t have enough time.

If I had only one more day.

These are some of the phrases that I hear on a daily basis in regards to time. It seems like the days pass too quickly and there is never enough time to get everything done.

Haven’t you wished a day could be 26 hours long?

In most cases, however, the problem is not that the day is only 24 hours long. The problem is how you spend your productive time.

Time Is the New Currency
Antique clocksI recently wrote a Simple Programmer post about how to Protect Your Productive Time, in which I covered many of the reasons why most developers — and many other workers in knowledge-based industries — can’t seem to get enough done in time.

In most cases, the problem lies not in the amount of work they have to do, but in how they choose to invest their time.

How much can you do in a year? I consider myself a very productive person. For example, over the past year, I worked on enterprise search consulting with Solr and SolrNet; ran my own small support center; managed my micro hosting company that leverages AWS, Azure, and Rackspace; created a few training modules for Pluralsight; wrote a book for SyncFusion; delivered a few public speaking engagements and training for the Atlanta.NET User Group LIDNUG and Search Technologies; led development of my SaaS solution for used car dealerships; developed the next version of my VIN application www.consultesuvin.com; and, on top of this, continued being a dad of two extremely active little girls, as well as a very helpful husband.

Nevertheless, I’m put to shame by John Sonmez’s feat of creating a staggering number of training modules for Pluralsight–55 of them–in only a year and a bit more.

As a fellow trainer with John, I attended an open space with him this year at Pluralsight’s author summit, where he walked me and a few other authors through the steps to supercharge programmer productivity. To be honest, I was impressed.

But being impressed hasn’t helped me much. Rather, it’s been taking John’s advice to apply my own personal experience to meet day to day challenges that’s made all the difference for me. (For those of you who have not yet attended an open space with John, you can learn his method in his recently released course: 10 Steps to Learn Anything.)

So, let me offer you some tips and point out some areas of potential improvement for increasing your productivity. One of my main suggestions in my previous post was to minimize meeting time and instead focus on getting things done.

This was a tricky proposition, as meetings cannot just be skipped altogether. Meetings can be a powerful way of reaching consensus, communicating progress, and, as per the cliche, getting everyone on the same page.

I’ve also written a few times before on this topic, for Pluralsight’s blog on tips for meetings, for my personal blog about my take on meetings, and how to make your meetings rock.

However, I wanted to write this follow up post to cover how to improve productivity in your day to day meetings in more detail .

Meetings: The Green Eyed Useful Monster
Meetings are a double edged sword. They can be extremely beneficial, or they can be a great time waster.

Indeed, meetings are so consistently abused that my Outlook often cringes in pain!

Sometimes I’ve felt as if I’ve been party to a meeting inception. Have you ever seen the movie Inception where people get in other people’s dreams and then go one more level into the dreams within dreams? That’s how it makes me feel!

In some of the projects that I have participated, it even feels like we have meetings to plan for meetings!

Eternal déjà vu. A glitch in the Matrix?

Most commonly, meetings are called to discuss issues that might not yet merit a meeting. In these scenarios, attendees are typically confronted with a set of unclear objectives, which are presented to far more people than are actually needed for the proposed initiative. Let’s dissect this idea.

Inviting Unneeded Talent
How many times have you been in a meeting and you see people staring directly at their laptop typing from time to time, appearing to be in their own personal bubble? What are these people actually up to? Three possibilities come to my mind:

They might be watching Facebook/Twitter/CNN/Insert-Your-Personal-Time-Waster-Here.
They are working. They had been trying to get stuff done, but they were summoned to attend this meeting, where they may or may not be needed. However, as we all know, declining meetings can be thought of by some managers as impolite, or even a sign of “not being committed to the team.”

They are active in the meeting, and they are trying to find a valuable piece of evidence to present to the team.

What percentage of the time do you think each one occurs? I leave this to you to decide, but I’ve arranged it to descend from most likely to least likely.

Boring presentation. Group of young business people in smart casual wear looking bored while sitting together at the table and looking awayConsider this. Time goes in parallel in meetings. Think of it in terms of billable time. If you summon 10 people into a room for a 1 hour meeting then it is not a 1 hour meeting. It is a 1 x 10 hour meeting. You just took away 10 hours of productivity time, or 10 billable hours, from a project. Let’s run some hypothetical numbers just for fun. If each person in that room is billed at $165 an hour, then your meeting just cost $1,650! You could’ve bought a new Lenovo Carbon X1 laptop with that time! And that’s a pretty nice laptop, I must say.

How do we avoid this kind of waste? The first step is for the meeting organizer to invite only those that are absolutely required. Sometimes a general distribution list is set up, for example the Team Leads DL, and everyone is invited. This may be necessary for weekly checkpoints, where the meeting’s purpose is general review to make sure everyone is in sync. For more specific issues, however, you should involve only the people that have something of value to add in that specific meeting.

And how do you determine this? Well, it is easy.

Clearly Defined Meeting Agenda and Objectives
Meetings are sometimes used by some as a way to appear busy. For them, attending a meeting is working. Do not fall into the trap of imagining an equivalency between meetings and work. Attending the meeting and being busy aren’t really the same as being productive. This mistake is common among underachievers who are big talkers. You may have noticed this group often includes managers. We all know a few.

Before you select who to invite, you need to have a clearly defined meeting agenda with a list of objectives.

But more important this, ensure that it’s crucial that the meeting take place. If the objective is not immediately important or required, then simply defer it until it is the right time.

A meeting without a clearly defined agenda typically ends up going on a tangent. Time is needlessly consumed and, very often, a follow up meeting is required. There’s another potential laptop lost to a useless meeting.

A good tip, especially when you are starting to make meeting objectives very clear, is to use a board for tracking the progress of your meeting’s objectives, ideally a Kanban board. It helps track progress so that all objectives are visually clear and a feeling of productivity is felt as you move along.

Scheduling Meetings
Argumentative persuasive businessmanBe sensitive to your team’s needs when you’re preparing to schedule the meeting. I understand how hard it is to work around everybody’s busy schedule. Depending on your specific role, you will view time in a unique way. A developer needs uninterrupted focus to be productive and create, whereas a manager’s responsibilities involve knowing what the developers are working on, so managers tend to view time in a different way.

Nevertheless, it is necessary to find a way for both of them to work harmoniously and get all their stuff done. This is why meetings are essential to meet production goals.

When it comes to scheduling meetings, try to find a time that allows for developers to maximize productive time.

A good strategy is to schedule meetings at the start/end of the day or before/after lunch. This protects blocks of productive time in the middle of the morning and afternoon.

Which brings me to my next tip: establish a working agreement to protect core working hours.

This is a practice that many organizations employ, wherein a team sets a schedule determining when meetings may be held. The schedule is respected both by the team and other bodies within the organization.

Here’s an example of a working agreement:

Big Data Inc. team’s daily scrum is at 8:30 am every day. From 8:45 am to 9 am all emails and inquiries will be responded to. Then, the team goes into core working hours from 9 am to 11:30 am.

During core working hours, all team members will focus on work items and bugs based on priority, critical, and high defects first. Important features are next in line. Further, during core working hours, developers will focus 100%, leaving IM, emails, and meetings until after core working hours are done.

Additional focused time can be scheduled individually by team members in the afternoon, priorities withstanding. The only exception is when there is a critical emergency that needs to be addressed and can’t wait. For example, a production down incident, in which case only specific and required team members may be required to help

Also be mindful that even during non-core working hours, you should interrupt developers as little as possible. As I said it before, developers need blocks of uninterrupted time to design/create software or fix bugs. Conversely, managers tend to see their time as 30 minute or 1 hour blocks that are perfect for scheduling meetings.

For the sake of both ensuring productivity and making sure everyone is working together, a balance must be created.

Talkers vs. Doers
We’ve all worked with very different kinds of developers. We all know the guy who is really good at talking, but usually doesn’t work much and instead gets others to do his work.

Then there’s the guy that sits quietly in a corner not saying much, but works like an ant.

When both attend a meeting, it will very likely be the Talker who takes the microphone and rants for a while–often about things that aren’t relevant to the meeting. At this point, the guy who needed to talk, the Doer, sits quietly in a corner.

How does a meeting organizer manage these different types of people to ensure they’re contributing meaningfully to the meeting and getting the most out of their coworkers’ contributions?

Well, first of all, you should have an agenda, so stick to it to limit the potential for others to go off on a tangent.

And second, on each point of the agenda, the meeting organizer must make sure that all parties involved in the issue under discussion speak their mind.

This can be tricky, but it’s fully achievable.

Action Items and Objectives
When your meeting is over, your team should have an actionable outcome or have made a decision regarding the points under discussion.

If a decision can’t be made over one or more meetings, you might be going directly into “analysis paralysis”.

I’ve been in that frustrating situation where repeated meetings never brought the team closer to a solution. It was like being on the set of Groundhog Day, but in a billion dollar corporation! Again, a lot of everybody’s time is being wasted.

If you keep having the same meetings again and again, reinforce the advice that I just offered and escalate if need be. Rinse, wash, repeat, and tame the green-eyed useful monster: meetings!

Do your best to make progress and move forward.

Summary
man points with fingers in the right sideLet’s summarize some of the points that I covered:

There never seems to be enough time to get everything done. The problem in most cases lies not in the amount of work, but instead how people choose to invest their time.
If you manage your time well, you can be very productive. But no matter how productive you are, there is always the potential to be more productive. Increase your productivity by protecting your productive time.
Meetings are a double-edged sword, as they interrupt your productive time, but are also necessary to make sure everyone is working together efficiently.
So whenever a meeting is required, invite only those that are most necessary.
Make sure you have a clearly defined meeting agenda.
Schedule meetings at times where they don’t interrupt the productive time of developers.
Get everyone to speak to the points that are of concern to them.
And do your best to have an actionable item, or make a decision by the end of the meeting.
I wish you a very productive time inside and outside of meetings!

 

Thanks to John Sonmez for the opportunity to write in his blog. If you want to supercharge your carreer, follow him!

This post originally appeared in https://simpleprogrammer.com/protect-your-meeting-productive-time/

Failing at Entrepreneurship While You Are Young Is Great

by Xavier Comments: 2

My Perfect Job

It was early 2006, and I had what most people would consider a perfect job.

I worked as a Developer Platform Evangelist for Microsoft Corporation, delivering labs worldwide to Microsoft Partners and teaching how to migrate applications to 64-bit computing. I was working as a contractor, commonly called a v-(dash) in the Redmond/Bellevue corporate lingo.

My life was great, and it was also great for my main employer. “Rent-a-geek” as I call it, is very profitable given the right customer, and we always delivered to Microsoft on time and on budget.

One week, I was in Silicon Valley, then London, India, Seattle, Sweden, Boston, the Netherlands, Korea, Building 20 (the old main campus training center), and the list goes on. If you are curious, you can take a glimpse of my globetrotting in my post, Road Warrior.

The usual joke with my fellow trainers was, “What continent are we in today?” We lived in an eternal state of jet lag, but who doesn’t want to travel the world, all expenses paid, while doing something you love? It was the perfect job.

Then I quit my job to go headfirst into entrepreneurship.

I failed.

And it was the best thing I could have done in my life.

The Opportunity

Good newsIt all started when I received an email from an ex-Microsoft employee who was starting his own company and asked me to help him with a potential project he had. I traveled to Seattle and was interviewed by a project manager (although if you have worked at Microsoft Corporation, you might notice that everybody seems to be a project manager).

The interview went great, and the project was approved.

That same night, I got a “formal” proposal from a hotmail address. A very nice rate for a three month project. Three months! It seemed like a tough choice and a risky one as I had a perfect job, and I’d been working for more than five years with the company that gave me the opportunity to work with Microsoft.

On the other hand, I had an offer for a short term contract that might be extended and turn into something better… Or maybe not.

Did I mention that, in my home country of Costa Rica, severance packages are mandatory and pretty high, unless you quit? It wasn’t an offer to start a company; it was a leap of faith to do a project, and if it worked we could take it to the next level.

I also had a hunch that this guy may not be the best choice. I ignored it.

What I learned: A hunch or gut feeling is your subconscious detecting a pattern that you can’t yet explain. Listen to it, but consider all factors as sometimes making the wrong choice can be good for you as well.

Letting Go of the Perfect Job

And I did it anyway. I picked up the phone, called the Human Resources department, and told them I was leaving.

Did I fear failure? Of course, but not enough to hold me back.

I also learned: To move ahead in life, you need to take action. John Sonmez said it very nicely in his Soft Skills book. If you start moving, it is easier to correct your course because you will have momentum.

Sometimes, you overthink too much, commonly called analysis paralysis, and opportunities pass you by. More than opportunities, life passes by, and you may wake up one day and regret everything you had an opportunity for and your own inaction.

Depositphotos_64017541_m-2015Human resources tried their best to keep me in the company. Just as an interesting note, I’d been asking for a raise for a while, and they always said no. They were probably very confident that no one would leave the perfect job. They offered me heaven and earth, including doublingmy salary. That actually made me want to leave even more as it just reinforced how valuable I really was to them.

I sent in my resignation letter that same day.

It was a very direct and friendly Dear Company, I quit. Thanks a lot! letter. I still had a good relationship with them, so no bridge was burned. I was counting on it in case my contract was not renewed after three months, forcing me to crawl back with my tail between my legs to ask for a job at half my leaving salary.

Something I learned: Leverage is really powerful. You might be in a position where you are 100% sure you are being downplayed, but if you don’t hold enough bargaining chips, you might not come out with flying colors out of the negotiation. I am not saying you should quit to get a raise, but it has worked for me twice in my life.

And a word of advice: You have to be also very careful of not falling completely on the other end of the spectrum. Some people believe they deserve more than what they have earned. I’ve seen this trend especially with very young developers who think they will land a job at Facebook or Google right after graduation and make several hundred thousand a year.

You first need to work hard to reap the benefits. You need to earn it first.

And then my adventure began at age 26. I took a plane straight from Boston to Seattle without going back home to Costa Rica.  All the time that I was traveling, I was using a corporate credit card, which I had to send back.

I only had $300 in my wallet and a $500 limit on my personal credit card. Finances were tight! I was young and totally used to swiping my corporate credit card and sending an expense report. This was a totally different experience as I was on my own.

I landed in Seattle and rented a car for a week (goodbye $500 limit on the credit card) and stayed for the first few days in a $29-a-night room near the Seattle airport. In theory—cross my fingers—I was going to get a cash advance from the new project and then get all the details sorted out.

About one week later, I got a $1000 cash advance, and soon after, all legal and financial details got sorted out. I could finally move into a shared apartment near Lake Sammamish in Redmond, about two miles from Building 120 where I was going to provide my rent-a-geek services in Microsoft’s main campus.

The cheap hotel was one hour away, but it is impossible to get anything cheap in the Bellevue/Redmond area.

What I learned: Sometimes getting started means you have to take a chance or two, and you will probably have to make sacrifices and compromises. Measure your risks, take action, and hope for the best.

I Am on My Own

I started working with Microsoft InfoSec, helping secure high business impact information. This was the three month project. It went well so I got another project, this time with Microsoft Consulting Services creating a SharePoint portal for CEOs and business people. I participated in a few more projects until, one day, I had a talk with the person who hired me.

At this point, I was providing consulting services on a one-project-at-a-time fashion, but he got me thinking: what if I went back to Costa Rica, and we started a company together? It sounded like a good opportunity to scale and get the company to grow.  We were going to be business partners.

What I hoped for: For this job, I got paid per hour worked. Creating and scaling a company (even at small scale) means getting people to work for you and getting paid for the hours they work, not only your hours. And I knew that I wanted to create my own company.

Starting The Business (The Second Opportunity)

People Working with Photo Illustrations of Startup BusinessIt was another opportunity, so I took it. Without telling anyone, I booked a flight back home. I remember getting to my parents place to say hi, and they asked me with a very surprised face, “What happened?”

Nothing happened. I just took a chance to create something bigger.

A few months later, I bought a house, and it was still empty; thus it became the first company office. Two floors, three bedrooms. I lived in one bedroom, and the other two became offices.

In a stroke of luck, I had two friends who quit their job recently, and they were working on their own dream—game design—but needed work to support their initiative. So they joined as well, and one of them is the best developer I know. He was known as TOC, or table of contents, as he always had the answer for your question. I still admire his skills to this day and hope to get the opportunity to work with him again.

We started to scale.

My now business partner who lived in Seattle was in charge of selling, managing US financials, and getting contracts. I was in charge of delivering, hiring, Costa Rican finances, legal work, dealing with bureaucracy, programming, project managing, pre-sales, delivering, presentations, demos, and pretty much anything required to grow the company.

I was basically the jack-of-all-trades swiss-army-knife guy for a while. In fact, I hired an assistant until we hit the 12 employee mark and an office manager at the 20 mark. The office stayed at my house until we had 8 employees. I took the “he lives at work” phrase literally. Even days when I had parties at home, I always had to remind my guests not to use the servers to place food or beverages!

What I experienced: Unless you are a funded or backed up startup, it usually means that you may have to do all kinds of work. I was president, janitor, and everything in between.

Growing the Company

The company kept growing, projects were delivered, and we kept hiring. It was quite a challenge, more than I am sure my business partner ever realized. We had a couple of problems.

Our first problem was that we were a small startup company where the office was in the 26-year-old president’s house in a third-world country ,and we were providing services to Microsoft Corporation. If I brought in someone for an interview, it may sound a little bit farfetched. If we were providing services to one of the biggest companies in the world, shouldn’t we also be in a fancy office?

Not quite.

People tend to forget how some big companies start in a garage. When you are getting started, it is critical that you plan ahead and save as much money as possible. Contracts and payments from customers may delay, but payroll must never be late. You should never ask your employees to wait for their paycheck!

Something we did right: If you keep your burn rate low, it gives you more runway to scale and adapt. Extra cash means being able to hire employees before you need them, allowing you to train them and get them up to speed instead of hiring them when there is an urgency and throwing them to the wolves to fend for themselves.

So many projects fail because people are not trained properly, and that’s why online training is so important, as Pluralsight can indicate.

But the first problem did not come alone. I ran face first into Costa Rican culture. In my home country, the employee mentality is rooted deep down in most people. Entrepreneurship is not the most common way to go.

Instead, everyone wants to get a job, work their hours, get a paycheck, rinse, wash, and repeat. When I was hiring, I had to inspire people into joining this small company that would grow and do great things! Come join me! You can work on world class, bleading edge SharePoint or Dynamics CRM projects!

Whenever I was doing an interview, if I liked the prospect, I had to switch hats and sell him or her the idea that we were what they needed to get ahead in life. It was tiring, but I did a very good job as I hired some great guys.

Brilliant ideaWhat I learned: If you inspire people, you can get them to work for more than just a paycheck. They can join you in a quest.

I also had to pull a few Houdini’s from time to time. Two weeks before we got our first real office, we received an email that a Microsoft Director of ISV Marketing and the CEO of a Microsoft Partner were in town and wanted to see the office. Crap! Which office? The unfinished one or my house full of cables and servers?

Also, being young, I had two big, off-roading, vintage Land Rover cars with no windows and an oversized, lifted Dodge Ram that was in no way fit for carrying around a couple of execs. I solved this problem with flying colors by:

  • Asking my uncle for his nice executive style car to tour them around
  • And instead of taking them to the office (or my house), I took them around the Free Trade Zone office complexes where Intel, Amazon, HP, Sykes, Microsoft, Boston Scientific, and many more transnational corporations have their technology and support centers. I made such a good impression that the CEO of the marketing company opened an office in Costa Rica which now has about 200 employees.

As a side note, Costa Rica used to be big on coffee and tourism, but now it has evolved into technology and call centers. After my initial failure, one of my next ventures includes owning a small call center, and my US customer is delighted with our level of service!

What I learned: Do the best you can with the resources at hand. You will be impressed at what you can do with some imagination and careful planning.

Realizing I Made a Mistake

Time went by, and that’s when I realized how I made a mistake.

When we decided to open a company in Costa Rica, we never discussed what was my percentage of ownership. We talked about ranges but we never decided an exact amount, and worst of all, we did not put it clearly in writing.

You see, when we started, it was all very fast, so we started under my Costa Rican company with an aim to create a new and fresh company once the startup took off. This could’ve been potentially disastrous to me if the startup failed as debts and employees were solely my responsibility.

As a real world example, one of my college professors started a game development company with some people from California, and when the company failed, he was personally liable for all debts, which he will probably have to work for 150 years to repay.

Luckily for me, that’s not what happened, but it could’ve ruined my life early in my twenties.

What I learned: Just like in the story of the chicken and the pig, be careful if you are committed while others are just involved. Take risks, but calculated risks. Share financial responsibilities with the others involved, and always make sure you know what the worst case scenario can be.

The Mistake

When it came time to take it to the next level of creating a fresh new company, that’s when it bit me in my behind. As the percentage of ownership was not clear, I couldn’t negotiate more than 5%, when I was personally responsible for the development success.

I am horrible at sales, but without my development lead, the company would not have grown the same way. If you need some help in estimating your percentages, there is a great method called “The Founder’s Pie” that I recommend you take a look at.

What I learned: You need to put everything clearly in writing as early as possible. You need to think of any possible outcomes and be in a position that you are comfortable with. My grandfather was a very prominent politician, and he used to say, “I trust you, but sign here first.” Sadly, I failed to follow his advice.

Something else I learned: Don’t ask for too much, but don’t undervalue yourself. That’s the part I always used to fail at. I usually undervalued myself too much. And yes, it is hard to estimate what you are worth.

Karma

Depositphotos_48589665_m-2015But how am I so sure of my value? It is very simple.

I already had a few difference of opinions with my business partner, but the one I think was the trigger of me leaving was that, in a Starbucks in Bellevue circa 2008, we negotiated what was going to be my end of year bonus as a partner in the company if we reached a certain amount of sales.

Our agreement was clear to me. He offered three bonus levels based on company performance. Given that I trusted him, we didn’t put in writing the exact scenarios with amounts. I flew back home hoping for the big paycheck next year when we met our objectives.

As you probably are already expecting to read, when the year was over, we met and exceeded the quota, but my partner used the “we need to save cash” excuse to avoid paying me what I was owed.

He knew the number, he knew I was right, but in an episode of undervaluing myself, I accepted only 10% of what I earned in all rights.

But it wasn’t the crisis. Now that we were profitable, I think greed took over, and he realized that he didn’t want to share the profits with a 20-something even though I was the reason the company flew.

I later found out this was his third attempt at creating a company; the previous two failed.

So I decided this was the end. It was Friday morning, and we were having a heated conversation. I just told him, “Find someone else,” and hung up.

This was probably the most stressful couple of days for this guy since I just hung up and never took any more calls from him, being as upset as I was. And even though I was the owner of only 5% of the company, I had unlimited power as president of the company—and did I mention we had a quarter of a million dollars in cash in the bank?

With this money and current contracts, my full bonus was not that significant. The story could be totally different for everyone involved, but it wasn’t. As I left the building, I just logged into the online bank manager, paid myself the last two weeks salary, and took off. I didn’t take a cent more, not even what I was rightfully owed.

Word of advice: Be careful with money. I could’ve taken my bonus and severance, but that would’ve probably ended up in a legal battle and I did not want to take it to that level.

The Beginning of the End

My business partner flew from Seattle, and negotiations were very sour. I still owned the domain and access to a lot of things that he didn’t even have. If he didn’t get everything I had from the company, it could be badly affected.

But this is not only a company.

It was a group of 24 hardworking individuals and their families. If I did something to hurt the company, I would also be hurting them. I ended up negotiating with one of my initial hires and gave up everything for a fraction of what my shares were worth.  It was a big loss for me, not only in terms of money but also because I loved the people I worked with in Costa Rica.

What I did: I chose to lose to avoid affecting those I cared for. Don’t be evil. Karma will take care of you later in life, so don’t be the one causing damage. And always keep a cool head and avoid your impulses. You are a master of those words that you keep but slave of those that you say.

Starting From Scratch… Again

This was me starting from scratch again, just like that day I took a plane to Seattle to stay in a $29-dollar-a-night hotel.  However, my situation was different now since I was engaged. I was on my own then, but not now.

I left the company on February 13, just one day before Valentine’s Day, and I was going to get married in 4 months. All preparations were ready for the wedding, but now being “jobless” and really tired of the drama was totally different than 2.5 years ago when I was excited to start a new adventure.

What I learned: When you are down, it is time to pick yourself up, shake your shirt, roll up your sleeves, and move forward. And so I started my new journey.

The Effect of Me Leaving the Company

Depositphotos_62880447_l-2015I mentioned before how I thought I was one of the main driving factors for the company’s growth. Why am I so sure? As soon as I left the company, it slowly started to downsize even though it had some of the best developers I’ve worked with. The company went from 24 employees down to 3 after about 4 years, at which time it was salvaged (acquired is a more friendly term) by another company.

So the reason it was moving forward was simple: I led the company to deliver results. Remember, delivering matters.

And so I started the next phase of my life at 29. It took me a day and a half to find a contract, not a job, and I have moved forward since then. I became a Pluralsight author and an author for Syncfusion Succinctly Series; run my own blog; own a small Support Center; created several cloud applications (success in progress, as I call them); have spoken at several conferences and meetups, including Microsoft’s SharePoint Conference, Atlassian Summit; and have been working as a consultant on multimillion dollar enterprise search projects with Apache Solr and FAST in addition to other areas like JIRA Agile, and Microsoft technologies.

Life has been great, and I am thoroughly enjoying every step of the way. I am working on things that I love, and it keeps getting better. If I think about it, had I not made the mistake of leaving my perfect job that spring of 2006, I would not have experienced all the adventures that I’ve had and would probably be working for the same company from 8 to 5 on a different project.

This mistake taught me how to run a company, and more importantly, it taught me what I want to be in my life.

Let me finish by adding a few more things that I have learned along the way:

  • Take opportunities in your life. A single email I sent out of the blue is how I got started as a Pluralsight author. That’s how I met John Sonmez.
  • There is no shortcut or easy path—at least that I am aware of. You need to put up the hard work, and it becomes easier as you get more momentum. Yet a lot of people spend their nights watching TV while saying what they want to do. Stop saying and start doing!
  • If you start young and with less responsibilities, you can take more calculated risks. Now I have two daughters with whom I love spending time with. But this means that I start working later every night, usually from 9 pm to 11:50 pm. I do it strictly Cinderella style—that is, I am in bed before midnight. And this is on top of my 40 to 45 hour work week.
  • Entrepreneurship is a marathon, not a sprint. I have this rule to work strictly until midnight because if I stay up too late, I’ll be next to useless the following day.. You are going for the long run; don’t burn yourself too quickly. But this is my biological clock. Yours might be different. And yes, I love so much what I do that sometimes I want to keep working, but a rule is a rule.
  • A failed project may turn into something good. I tried to set up a reseller service for an automotive SaaS (software as a service) product from a US corporation in Latin America. We failed, but this lead to an opportunity to set up a call center for this company, and I have been providing this service for three years now.
  • Every decision I make and every action I take is a stepping stone to becoming a better person and a better developer. I am working hard at mastering my craft and gaining control of my life, which brings a question to my mind: are you in control of your life? I recommend you read this post as well.
  • There is one constant in life: it goes on. Don’t live in the past, but learn from your mistakes.
  • And making mistakes is ok in most cases. There is always something that you learn out of it that can help you in the future. A mistake can be success in progress!

Since the day I failed I have been working hard day and night to create my dream. Yes, everybody dreams, but only those who take action, make a plan, and move forward get ahead.

Be positive. There is a sea of opportunities out there but only for those willing to devote themselves entirely to making them happen.

 

 

This post originally appeared in: https://simpleprogrammer.com/failing-at-entrepreneurship-while-you-are-young-is-great/

Protect Your Productive Time

by Xavier Comments: 0

You only have 24 hours a day…1,440 minutes…86,400 seconds. At first glance, this may seem like a lot.

But it is not.

There is a very limited supply of time, and it can be your friend or your enemy. And you get to choose if you are in control of your time, or if you let others control it.

That is why you need to Protect Your Productive Time.

But why? Why do you need to protect your time? And how do you do it? Let me tell you.

I am a .NET developer who is very passionate about enterprise search, primarily with Apache Solr. And this is great because I’ve spent the last few months of my life building the search API library for one of the Big Four auditing firms. It is a huge project, with hundreds of developers, and my library is only a very small piece in comparison to the rest, but it is a very important piece none-the-less.

And while working here, one of the things that I’ve noticed is how programmers have a tendency to give estimates that are not too accurate. Why does this seem to be a recurring issue?

Well, there are the endless meetings that they don’t account for in their estimates. There’s their own overconfidence — or ego estimates as I like to call them — and a tendency to assume features are simpler than they really are. Also, most people need a “warm up” period in the morning to get into the zone or “flow,” but then they have constant interruptions. If you want to learn more about getting in the productivity zone, I recommend Mihaly Csikszentmihalyi’s book Flow. It’s a classic and a must read.

Going back to estimates, let’s say for example that you estimate a task will take 40 hours, so just in case, you overestimate and say it will take 60 hours. This won’t work. Why? One good reason is that the next 20 hours might be filled with another 10 hours of meetings and interruptions. Just think about it for a minute. How many times a day (or hour) do you get someone that comes to your desk and says “Hey, do you have a minute?” and in some cases without even letting you answer, they sit down or put a laptop on your desk and start explaining what they need.

Say noSo let’s go back to the original question. Are you still going to be able to be done in time?

The answer is usually no.

So what I observed is that some people couldn’t function properly in this environment and couldn’t deliver what they promised when they promised.

Not me.

I delivered what I promised.

On time.

With very little bugs (I can’t lie. I had bugs!).

How?

Was it because of my technical prowess?

Not really. I am a decent developer (and can proudly say a Pluralsight author, Syncfusion author, passionate about Search, speaker, presenter, and trainer among a few other things), but nothing out of the ordinary. There are some developers that I know who are way better than me technically, but I can beat them by a mile in terms of delivery.

Here’s the “secret sauce”:

I did three things:

  • Communicated properly
  • Organized myself
  • Protected my productive time fiercely but kindly

And I was very aware of the first two, but not too much of the third one, which now that I am fully conscious of its importance, I put it as a priority in my day to day work.

When and how did it become very evident to me? Here is the exact moment.

So, there I was one day minding my own business, heads down in Visual Studio, when Brent (one of the testers) approaches me and asks if I have time to review a bug and explain. It was a good moment — that’s what I thought — so I said yes. So he then looks to his side and about 3 cubicles down he signals a thumbs up to Radhika, another very nice tester, to which she smiles and with a face like a kid on a christmas morning says “yaaaaay”.

Depositphotos_55381575_m-2015(1)And then it hit me.

I had been protecting my time by managing all interruptions in a way in which I could focus on working and then schedule time for other people’s needs later. This might sound very simple, but it wasn’t. Everyone has their own deliverables for a specific date and time. And as much as you would like, their deliverables are their top priority, not yours.

And the better you are, the more time people take away from you. I have a friend who peers call “Table of Contents” because he knows everything. How much time do you think he spends working on his deliverables vs. fixing issues on other people’s deliverables? The answer is simple. Most of his time is spent helping others, and he has to work until late at night to finish his stuff, while the developer he helped goes home at 5 pm.

To add insult to injury, top developers usually get more work assigned. It kind-of makes sense from a management perspective. If you have a developer who is very productive, assign them more work and overall the project will benefit from higher productivity. The problem is that this is just short term thinking. As you squeeze out every last possible drop of code from a top notch developer, the end result will usually be a burned out developer or a resignation letter.  Some managers don’t even seem to care. Burned out? Get another contractor or employee, rinse, wash and repeat. But this is a discussion for another post.

Let’s get back to time. Does it sound familiar that others come to your desk asking for your time? Do you have to help them the minute they arrive because “it is urgent”? Or, “C’mon man, it is only 5 minutes”?

What most human beings forget or don’t know is that the human brain requires concentration and focus, especially for activities like coding or design. Context switching breaks your process, which means that you need time to get back into your thought process. I am not an expert, but I heard that from any interruption you need at least 15 minutes to refocus and keep working. So a 1 minute interruption in reality can be 16 minutes that you will never get back. That’s why John Sonmez asks people not to message him while he is focused at work.

In my humble opinion, that is not a good deal for you.

Do you just put on your noise cancelling headphones and a big “Do Not Disturb” sign in front of you and code away?

It is not that simple either.

Unless you are a one man army product development unit, or your work is totally isolated from everyone else’s, then you are a necessary and useful component of your team’s development, and there is an intricate set of dependencies that cannot be ignored.

So what should you do? Let me tell you what I do and a few more tips that I have learned while out on the field.

  1. Turn off chat and rely on an asynchronous communication method. In layman terms, instant messaging applications like Skype, Skype for Business, Lync, Google Chat or similar will keep breaking your focus continuously. I do agree that they can be tremendously beneficial, especially for distributed teams, but in most cases chat can be more of a distractor.
    But don’t uninstall it right away. You can use the “Busy” status so that you are not distracted while focusing on your work, and then during break times you can go back, check the messages and respond. Most applications also let you set a special message, so you could add a phrase like “Focused work time. Will reply soon” to let others know that you are not ignoring. This is very useful, primarily if you are working in an organization where it is common to expect prompt replies to IMs.
  2. Take breaks. Did you know that your brain needs breaks? If you want to learn more about recommended work times and breaks to increase your productivity, I suggest you check out the Pomodoro technique.
  3. Manage your email instead of letting it manage you. This is the other big distractor: email. We all get bombarded by emails on a daily basis. A friend of mine, and the best PM that I have worked with, Ian, had a personal game where he kept a record of how many emails he received on a given day related to the activity that was taking place. Kind of like when you do trend analysis on Big Data, but at a small scale. “Today I received 259 project related emails and it is demo day of version 1.7!”

If you consider how much time it takes for a human being to read 259 emails and respond to them, then he would spend all of his day just reading and replying without having any time to do any of his work.

I assume some of you already did the math, and in his specific case, he got an email almost every 2 minutes in average.

Yes, his was an extreme case. I’ve noticed that the higher you are in the “food chain,” the people have a tendency to add you to the thread. The developer can later claim “Ian was copied,” and that can get him out of trouble (or at least the developer thinks it will).

The same advice goes for email. While email is very important, your objective as a developer is to deliver working code within budget and on time. So please do not forget about email, but don’t make it priority #1 above your deliverables.

Remember, responding emails can keep you from getting fired, but delivering top notch quality code can get you promoted or a raise if you play your cards right.

I recommend you schedule email-dedicated times, and let your peers know so they have a better expectation of when you will be replying.

Cute child begging IT support for helpOkay. So at this point, we have covered two of the biggest distractors, IM and email. But what about the “hey do you have a minute?” cases where someone sits next to you and expects you to jump head first into their need.

Don’t get me wrong. I do it from time to time. It is an impulse. You need something and someone else has an answer. The impulse grows the more desperate you are.

Here is an example from a few months ago. I had already about an hour — that felt like an eternity — trying to modify a dependency injection scenario where I couldn’t get the bloody thing to work, and I knew that both of my team members, Sandeep and Satish, who were sitting quietly in their cubicles across from me, could definitively help.  First of all, I was trying to get it done by myself. Just asking for help at the tiniest issue is not good. As a developer, I get paid to deliver, which involves finding solutions to problems that I run into in the process.

I had it almost sorted out. There was something that I was missing. When I decided it was time to get help, I did interrupt Satish, but I did in what I believe was a nicer way, not being abusive of their time.

I waited a bit for a time when I believed it was okay to ask for help, which was after his phone rang and he hung up. So I approached him and said “Hi Satish, I have a problem with IoC. Would you be able to help me when you get a chance.” I didn’t take my laptop with me either, because that would be applying pressure.

He said: “Yes I can, please give me 15 minutes.” About 20 minutes later, he told me he could help. And indeed there was something I missed, I corrected the error, continued development, and was able to check in the feature a bit later.

I did interrupt his work stream, but in such a way that he commanded his time. I asked for his help, but on his terms, whenever he felt it was a good time.

As I said, there are dependencies that can’t be ignored, but the trick is to be respectful of other’s time and expect the same.

wireless internet connectionIM, email, and one on one help can destroy your productivity, but they are more manageable than the next level of interruptions that can wreak havoc on your productive time: meetings.

Meetings are a double-edged sword. They are an absolute must in some cases, but a complete waste of time in others. And it may be difficult to know in which direction it is going to go. So let’s take it one step at a time with a real world scenario.

I participated in a large Agile project. In such a case, there are multiple teams that have different focused responsibilities. At a general level, you have the business people coming up with the specs, which then flow to the designers, who then pass it along to the architect team, development, testing, and deployment. The process is a bit more complex than this, and it is called Daikibo or Agile at scale, but you get the general idea.

In such an environment, it is of utmost importance that there is fluid communication and that every team down the stream understands fully what needs to be done.

Thus, we have meetings. Granted that specific issues can be followed up via emails, work items or IM’d, but sometimes a meeting is required to — as the cliché says — “get everyone on the same page.”

I give kudos to a well-run and organized meeting that has a very clear objective, which is met, and only the right people are involved.

Does this happen often? No, not really.

Bored panel of judges or interviewersMeetings are abused continuously to levels that make my Outlook cringe in pain! I’ve already blogged a few times about this in Pluralsight, and on my personal blog about how to improve meetings and what hurts your meetings.

Sometimes I’ve felt as if we’ve had meeting inception. In some of the projects I have participated, it feels like we have meetings to plan for meetings! Eternal déjà vu. A glitch in the Matrix?

Increasing your meeting’s productivity is a topic that I intend to cover very soon in a different post, but for now, the general recommendation that I have for you is to minimize attending meetings to only when strictly required, by those that have something of value to add to the meeting, and at times that do not interrupt those who need uninterrupted time to concentrate and work. An example is at the beginning of the day, around lunch time and near the end of the day.

This is a concept known as core working hours. I saw a team down the hall put up a sign that explains this very clearly: “Core working hours are from 9 am to 11:30 am. Please do not interrupt, and expect a delay on our responses.” How great can this be? 2.5 hours of uninterrupted work time. So much productivity behind a closed door!

At first it might be a bit hard to have a part of the team not responding to your emails right away, providing instant replies to your messages or attending back to back meetings. But there are some very clear benefits to this model.

First, makers will concentrate on delivering. Don’t you hate when there is a team member that has a high priority item that is urgent, they don’t finish, and when you ask them why they answer “because I’ve been in XYZ meetings and responding to my emails.” This drives me nuts!

But notice that I am not saying all developers do this. Only a few do, but you get a potential excuse out of the way. For the rest, they get time to concentrate and work.

Another benefit is that other team members, when they know that a person will not be available during a particular period of time, will get used to planning ahead and communicate using email or via comments in work items. It doesn’t matter that much which tracking system you use, but it does make a difference if you use one. In Simple Programmer, we use Trello, but I am a big fan of Jira as a work tracking system. So much so that I have two courses on Pluralsight on Agile, both with Scrum and Kanban.

A core working hours agreement will definitively allow team members to have time dedicated exclusively for working, and this is great. But, you do have to take it one level above, and this is the hardest one: getting each person to be as productive as they can be.

This is no simple feat in any way that you look at it, and it goes beyond what can be done at a team level because it needs to happen at an individual level.

For a person to reach a peak and optimum performance level, they need to be in the right conditions — core working hours being one that I want to convey — and in the right mindset.

How can this be achieved? There are multiple books and methods that can be learned to achieve maximum productivity. For example, Sonmez does it using the Pomodoro technique, which he covered in the Soft Skills book and released recently in a course on his productivity secrets.

But this is the topic of discussion for another post, which I believe will be touched on at length in many more Simple Programmer posts, as well as the book and recently released course.

What I wanted to focus on was how to set the right conditions in a work environment that will allow you to be ready for the next level of maximum productivity.

Stylish young businessman doing a presentationLet’s do a quick recap of what I presented:

  •  Time can be your friend, or it can be your enemy.
  •  You can manage your time, or you can let others manage your time.
  •  To be able to create (i.e., write code in case of a developer) you need blocks of uninterrupted time.
  •  After each interruption, your brain needs some time to refocus and get in the “flow” to produce optimum  results.
  •  IM is a constant interruption, try to minimize the use of it and set your status to “Busy” with a friendly message when working.
  •  Schedule email.
  •  Don’t interrupt people with the expectation they will help you right away; instead, ask politely for help on the schedule of the person that you are seeking help from.
  •  Meetings can be good, but they can be evil as well; thus, invite only the people that are absolutely required with a clearly defined agenda and meeting objective.
  •  A core working hours agreement can set the right conditions for levels of optimum performance to flourish.
  •  Time can be your friend, or it can be your enemy.
  •  With the right conditions, going to the next level is up to you.

And there you go, that is my advice. Protect Your Productive Time!

 

Thanks to John Sonmez for the opportunity to write in his blog. If you want to supercharge your carreer, follow him!

This post originally appeared in: https://simpleprogrammer.com/protect-productive-time/

SolrNet Release 0.5.1: Fix Spellcheck Parser Issue

by Xavier Comments: 0

SolrNet, the C# client for Apache Solr, has a new release: 0.5.1. The current release aims to include a breaking change with the latest versions of Solr 4.x in which multiple collations are returned by Solr. I am currently working on getting it to Nuget

This is the release:

SolrNet Release 0.5.1: Fix Spellcheck Parser Issue

Let me show you quickly with an example, here is how a single collation was returned before:
<response>
<result numFound=”1″ start=”0″>
<doc>
<str name=”Key”>224fbdc1-12df-4520-9fbe-dd91f916eba1</str>
</doc>
</result>
<lst name=”spellcheck”>
<lst name=”suggestions”>
<lst name=”hell”>
<int name=”numFound”>1</int>
<int name=”startOffset”>0</int>
<int name=”endOffset”>4</int>
<arr name=”suggestion”>
<str>dell</str>
</arr>
</lst>
<lst name=”ultrashar”>
<int name=”numFound”>1</int>
<int name=”startOffset”>5</int>
<int name=”endOffset”>14</int>
<arr name=”suggestion”>
<str>ultrasharp</str>
</arr>
</lst>
<str name=”collation”>dell ultrasharp</str>
</lst>
</lst>
</response>

And then with later versions of Solr 4.x multiple collations were returned:

<response>
 <result name="response" numFound="0" start="0"></result>
 <lst name="spellcheck">
 <lst name="suggestions">
 <lst name="produtc">
 <int name="numFound">1</int>
 <int name="startOffset">0</int>
 <int name="endOffset">7</int>
 <arr name="suggestion">
 <str>product</str>
 </arr>
 </lst>
 <lst name="collation">
 <str name="collationQuery">product</str>
 <int name="hits">1000</int>
 <lst name="misspellingsAndCorrections">
 <str name="produtc">product</str>
 </lst>
 </lst>
 </lst>
 </lst>
</response>

The detail was that SolrNet would raise an issue because the numFound node was not found. Well, this issue was fixed now.

This is just the first release that I do on SolrNet since I was granted permission to provide new releases. I am merely getting up to speed to work my way through the backlog of improvements and including support for newer releases of Solr.

If you have any questions, don’t hesitate to contact me via this blog or @xmorera in Twitter.

 

How to Install TeamViewer on CentOS (useful on Cloudera QuickStart VM)

by Xavier Comments: 0

The other day I needed to finish a task I had in one of my servers and needed to remote into one of my Cloudera QuickStart VMs to run a test while on a trip. So I installed TeamViewer to access it. Steps are simple:

# Click on Download TeamViewer link for RedHat, CentOS, Fedora, SUSE to get the rpm package from the downloads page

https://www.teamviewer.com/en/download/linux/

Open terminal and go to downloads directory

sudo yum localinstall teamviewer_12.0.71510.i686.rpm

And then start with

teamviewer

The Art of Creating Applications That Have Search

by Xavier Comments: 0

In my Pluralsight trainings, Getting Started with Enterprise Search using Apache Solr and Implementing Search in .NET Applications, one of the things that I make quite a bit of emphasis is on how important search is, yet it is one of the most misunderstood functions of IT and development in general. In this post I will show you an example of how a potentially good app is a pretty bad app mainly because of its search capabilities.

It is so much the case that in Twitter Pluralsight selected this phrase to tweet about the release of my course as you can see here:

searchiseverywhere

But now let’s get to the sample. Here’s the scenario:

Problem: Life is busy. No time to go to the supermarket

Solution: use your grocery store’s web site to purchase your food and it gets delivered home the next day. Charming idea, did not work with Webvan, but it seems to be doing quite well for Amazon and in my home town one of the major supermarkets is doing it in a more controlled way with a good delivery service, all for $10. Not too scalable, but for a MVP it is ok. (Read Lean Startup if you don’t know what MVP is)

It may work or maybe not mainly because of a really bad user experience, but let me get to the point. UX is important! Never forget it!

You get to the app in https://www.automercado.co.cr/aam/showMain.do and they have mainly 4 sections as you can see here

auto

And here is what they are for:
– On the left they have a directory style organized by aisle. Grouping kind of works in my opinion if you are not too sure of what you want, but it is terribly slow and inefficient. They lose cookie points for this.

2014-07-02_0638

– Then in the middle they have a section where they display the products. This is very standard so it kind of goes through, however they lose cookie points again for having products without pictures or with very weird stretching. They are a supermarket, and a big one, so I am sure they can send a guy with an iPhone to take a quick picture.

2014-07-02_0637

– The cart has a problem which is that they do not actually display the product name, only the description. Who thought of this? Not even something as simple as a tooltip!

2014-07-02_0640

And then here is the deal breaker for me: BAD SEARCH! As mentioned in the post, search is one of the most misunderstood functionalities in IT. A lot of people make huge mistakes because search can be done with a database, which it can, but the end results sucks! And it did suck here.

Let me show you this. I want to look for “jabon dial” which means “Dial Soap”. So I just type “Jabon Dial”. Should work, right? It doesn’t! Look at the message: “No results found…”. Also I hate the CAPS. There may be 1 technical reason I can think of but it is pretty dumb.

2014-07-02_0646

But why? If you look closely there are 27 types of “Jabon Dial”, type only Dial

2014-07-02_0649

The problem lies here:
– The person that implemented this application had no knowledge of how search works, which is normal as search is pretty misunderstood.
– But humans don’t do search like engineers want. Having the user do a search exactly like the engineer wants is just lazy and ineffective.
– So engineers who created this probably went for a simple exact match in a database search
– This is a terrible user experience. I can bet the farm that Amazon would have closed its doors in the 1990s if they had such a bad search

How to fix it? Well, go learn how to use a search engine. And that’s why I created my course, Getting Started With Enterprise Search Using Apache Solr: http://www.pluralsight.com/training/Courses/TableOfContents/enterprise-search-using-apache-solr

Tip of the Day: Disable Screen Blanking in Linux CentOS while Installing Cloudera CDH

by Xavier Comments: 0

Something that really annoys me, especially when connecting remotely is how the terminal blanks when installing Cloudera Manager in Linx CentOS

Well, there is a very simple fix, simply run the following command and the terminal will not go black

sudo setterm -blank 0

The Day We Started to Outgrow Relational Databases

by Xavier Comments: 0

Look around you. Look closer. Pay more attention.  What do you see?
When I look around me I can see activity trackers, digital cameras, smart watches, interconnected devices, virtual reality gadgets, wearable technology, smart elevators, energy saving light systems, intelligent traffic lights, smart cars with over the air updates and that can gather data on your driving habits, intelligent houses, eco friendly buildings and more.
All of these generate massive amounts of data.  But let’s hold that thought for a minute.
Now this is just what’s happening around you.  What do you have in your pocket or in your hand right now?
Most likely a smart phone. It is your portal to the digital world and even though it became second nature – pretty much everyone walks around with a phone in their hand now a days – it is a relatively new phenonem.
It is highly likely that you use your smartphone constantly to check Facebook, Twitter, Instagram or search the web using Google among a few other applications. This generates humongous amounts of data.
Let’s throw out a few numbers just to put it in perspective. Facebook has 1.6 billion users- yes, that is with a B – millions of them who log in every day to upload millions of pictures, add comments, like posts and perform many actions. And every action has an impact but as the amount of data grows, it gets harder to determine what that impact needs to be.
Then we have Twitter, which may be a tad smaller albeit still plenty of data by any definition. But the beauty of Twitter is not just the human interactions, but instead what can be extracted from the data.
And Google… well… what can I say? Try indexing the internet and then we can talk about it. Just do it, invite me over and I will buy you a coffee while you tell me how it went.
How much data do you think is generated daily by all these applications? But besides the applications, remember how I mentioned many devices that also create mountains of data? This means that besides human generated data, we also have machine generated data.
A Big Data World
The world that has changed in unimaginable ways. Together, human and machine generated data, bring us into an information explosion era the likes of which the world has never seen. You are living a digital revolution and you can consider yourself lucky for being part of it.
And tweets, posts, likes, pictures and stats are very nice. But that is just the tip of the iceberg of what’s to come.
There are many other applications that require analyzing those massive amounts of data to help reduce costs, detect fraud, and many other potential use cases that help drive innovation for all mankind. All these are scenarios that help increase profits, decrease costs, innovate or help stop the bad guys are nice. But let’s take it up one notch.
There are some people trying to make a difference like hospitals that are working to cure cancer by analyzing DNA records, comparing them and find ways to save human lives. Imagine if one of those lives they saved was your son, your daughter, your wife or your parents.
The world has changed around us. We now live in a world of Big Data.
Getting Insights
But data by itself is just data and as I mentioned something needs to be done with this data to get insights.  How can this be achieved?
Well, let’s first rewind a few years and think how this was done before. A while back, if you had a “massive” amount of data you went to your prefered vendor and wrote them a huge check for an equivalent machine. Then you wrote another very big check to your favorite database provider to run in this machine and built an application that could consume the data, process it and give you the answers you needed. Also, you usually had to limit the amount of data as you were constrained by the limits of your big box so you had to throw data away.
Outgrowing Relational Databases
But there were times when a big box was not enough. For example, what if you wanted to index the entire internet? There was no box big enough for this.
Also, there were a lot more scalability constraints like performance. If you had 1 terabyte and it took 1 hour to process and then you add another terabyte, well it will probably take almost twice the amount of time or even more.
And this was a problem that needed a whole new way to be solved. It all happened when Google published a paper circa the early 2000s where they explained how they invented a way to solve this problem using GFS and MapReduce.
And then magic happened. Two Yahoo employees, Doug Cutting and Mike Cafarella, read the paper and they had to solve the same problem so they got inspired and created Hadoop!
Welcome to Distributed Computing at Its Best
Hadoop took a different approach to solve any Big Data problem. Instead of a big box, it relied on creating clusters of many smaller and way cheaper computers, also known as commodity hardware.
The data is then distributed among all these computers and then processed locally. In hindsight it sounds so common sense, but instead of taking the data to where it is processed, it takes the computation to the data. Each individual node in the cluster has its own copy of the data and does all the computation locally.
Agreed, a big box is probably more reliable than a bunch of commodity servers, so probably a few of them might fail during processing. This is not a problem as Hadoop has data redundancy so if one server fails, there is a copy of the data replicated in another server that can pick up the work.
And so we have distributed, resilient, dependable and efficient Big Data systems that are helping us change the world.
Data drives the modern world. But who drives the data? That is what Cloudera is here for.
Ask Bigger Questions!

Big Data… Big Deal? Big Hype?

by Xavier Comments: 0

Big Data

Big deal?

Big hype?

Or big change in our world?

I think that the answer can be all of the above. “Hype” you might be thinking? Well, here is the deal. Our world has changed in unimaginable ways. The amount of information created daily is reaching levels that just a few years ago would’ve been considered science fiction or even plain old crazy.

Lots and Lots of Data
To make it even more interesting, a lot of it is unstructured data. Which can be kind of a problem if we think about it, because the success of relational databases has taught a lot of us to think in a columnar and relational way.

And this is not bad… at all. It is nice to have all your data and metadata organized neatly. You can use select, join, where, group by and more to get what you need.

But the success of relational databases can also create a blind spot for many. Just a few days ago I was talking with the VP and cofounder of a company related to migrations and artificial intelligence software whose company has faced success (as well as a few failures or learning experiences) in several world class projects. They had lots of data that they obtain from their automated code conversion tools and what are they doing? They are normalizing it into a database.

I don’t think it is a bad approach, however it is not the one that I would take. Long story short, I would store the logs as is in their raw format and then use any of the available projects to analyse it in multiple ways, looking for key points, failures, trends and more. But what you do with the data is the topic of another post or a Pluralsight training. Let’s go back to our main point.

Mountains of data is being generated daily and the amount will just continue to -grow- explode.

Unstructured Data “Just Happens” 
If you had to structure all your data, do you imagine what the cost would be? Just go ask your manager for an Oracle system and some servers to process all of your web server logs to put them in tables. The cost would be exorbitant.

And beside cost, sometimes you may not know the structure of your data. And that is one of the beautiful parts of Big Data. You can just store your logs in raw format and later come back and do your work, modelling your data in different ways. And what if you have too much data and the process is taking longer than expected?

Well, just add a few more servers and get the job done in parallel. Hadoop runs in commodity hardware, thus you can get many relatively inexpensive machines to work together and process your data according to your needs.

The Cloud and the Bar
And even better, remember “the cloud”! A few years ago if you were a startup and needed beefy power, you would need a lot of upfront cash to cover expenses. Now with AWS and Azure we have the possibility of turning a few virtual machines, get a cluster up and running, crunch the data, get the result, turn them off and only pay for the time you use.

And this change has lowered the entry bar for innovation. Now many brilliant ideas can be tested or theories can be analysed at a much lower cost, benefiting all man kind. For example, it is possible to run analysis on medical treatments to help cure cancer or many other diseases. Sometimes answers to hard questions lie right there in the data, they just need to be discovered.

Hype or Go Figure This Hadoop Thing Out
But what about hype? Let me make this clear, I don’t think Big Data is hype. I do think that there is a lot of hype around it and even though we are able to do great things with Big Data, the greater public does not yet fully understand what can be done and how so I have taken a personal mission to help developers and the public in general understand Big Data (and Search)

So then it is time to ask ourselves this question:

What Are My Choices for Getting Started with Big Data?

Big Data Links

by Xavier Comments: 0

Here is a collection of some interesting or fun articles that I have found on Big Data
5 Reasons to Move to Big Data (and 1 Reason Why It Won’t Be Easy): gives an easy to understand set of selling points on why to adopt Big Data, but making it clear some of the issues you might face.

– The Most Practical Big Data Use Cases Of 2016: covers some interesting use cases of Big Data. Remember, Big Data is sexy!

Why ‘Big Data’ Means Nothing Without ‘Little Data’: Little Data is regular performance metrics.

– Why Big Data is the new competitive advantage: provides good points on how Big Data can help give you an edge.

Big Data: What is it and Why it Matters: goes straight to the point to explain the basics.

Big Data Analytics: What is it and Why it Matters: explains what is Big Data Analytics.