As the field of SE/CS is getting more press, graduates are flooding the market. Yet, the curriculum given in many universities still seems barren when it comes to professionalism, forcing newcomers to learn via unpolished street creds. Not only is it leading to mysticism about what skills are required but is also leading to a lack of discipline, duty, and craftsmanship.
We would be tempted to assume that those graduates would be instantly
digitally/web literate and professional in their field, however, with the
ever increasing volume of knowledge stacking up, we’re at loss. What we’re
left with as a solution is the slow and painful process of passing the
flame via mentorship and deliberate practice. And to be fair, universities
may not be the place for such type of learning.
With this in mind, let’s review some of the topics and practices that are important to know. Let’s write what seems dull but that no one dares to express.
- How to Ask Questions
- Searching Online
- Roles in a Team
- Reading and Writing Documentation
- Working From Requirements
- Programming Paradigm
- Design Patterns and Modularity
- Best Practices, and Reviews
- Test units
- Storage Technology
- Configuration Mechanisms
- Package Managers
- Content Versioning System
- Continuous Development, Pipelines, Deployment, Fast Release Cycles
How to Ask Questions
It is easy to underestimate the value of asking questions properly. While in a class setting we’re eager to say “No such thing as a stupid question”, it’s not quite the same in the software world (Though most people asserting they’ll be asking a stupid question are the ones asking legit ones). Questioning turns into an introspective activity that is honed over the years.
Before asking a question, it is expected that a decent amount of
research be done such that it can be formulated without ambiguity or
misinterpretation and with confidence, basically knowing exactly what
you want to ask. That means searching the web, checking online manuals,
reading FAQ (Frequently Asked Questions), diagnosing and experimenting,
and even reading the source code if possible. A lot of common questions
have been answered at least once. With all this, the question should be
precise and concise.
Unsurprisingly, this is enough to answer the vast majority of questions. For the rest that can’t be answered alone, we have to try our luck and ask the appropriate persons. Finding them is a quest on its own, unless they are sitting close by, it could be a forum, stack overflow, an IRC channel, a mailing list, etc.. When the medium of discussion is found, the question should be formulated in a way that respects it, is easy to reply to, has an accurate title if requested, has no grammatical errors, is not overly verbose, and again, be precise and informative.
After posting the question, there may or may not be replies to it. In a lot of cases there isn’t and we should go back to research. This is why many foresee the complexity of the question they are about to ask, and the lack of experts on the topic, and prefer not to ask and to stick to research only, sometimes later on posting their findings in blog posts and articles. In case there are replies but with misinterpretation of the question, then the wording choice might have been inadequate and a reformulation can be done. And if someone replies with RTFM (Read the Fucking Manual) or anything rude, don’t worry this is the internet, do not take it personal.
A lot of questions get asked not because they weren’t researched enough but because the searching methodology itself wasn’t grasped. Even though most people use search engines everyday, not everyone knows how to use them when it comes to advanced subjects. Indeed, like any tool, there’s a skill set to learn.
The first thing to realize is that search engines are not omnipotent,
they cannot read minds, and they’ll always give the easiest and most
popular results for the query they are given. In sum, it’s a constant
battle. That means that full sentences are not interpreted if they
aren’t part of a preset. While sentences like “What is the weather
today” may work on some search engines like Google, it is the exception
rather than the norm when it comes to searching online. For anything
else the search engines usually split the terms by spaces, search each
terms individually, and return links that match as many as possible,
prioritizing them by popularity, sometimes respecting the order of the
words given, and sometimes not. So how do we cope with this?
To let search engines know what we want, we have to speak their language. For example, putting words inside quotes will tell the search engine that we absolutely want them in the results, prepending a word with minus will tell it to omit results that contain it, using keywords such as “site:” and following it by a domain will force the search engine to only have results from this domain. Those are ways to calibrate the search criteria.
In theory, with this method we can find what we want. However, what if we don’t know what we want, what if we’re unaware of the exact wording to use, what if what we want doesn’t match what exists on the web, what if what we want is buried under thousands of popular clickbait articles, or what if what we’re searching for is based on something we forgot and are trying to remember. Thus, we have to enter an iterative process with the search engine: We ask it something, check the results, change the query to get different results based on what we’ve got, and repeat everything until we find what we’re looking for.
This iterative search, surprisingly, is something not a lot of people learn, or are comfortable with, but that is mandatory in the tech world. Additionally, it’s this sort of procedure that leads to the discovery of interesting topics. Nonetheless, it requires patience and a hunch for knowledge, the so called “learning how to learn”.
Roles in a Team
A lot of early entrants in the field have the dream of becoming the
media fetishised lone-wolf developer superstar, fortunately, they
couldn’t be further away from the truth. In the professional context,
getting entranced by a condescending feeling about one’s own skills
is counter-productive. On the contrary, teamwork, understanding, and
humbleness, are what drive productivity forward.
Any project bigger than a certain size has dynamics that are better managed as a group endeavour. In well functioning teams, members are expected to handle their responsibilities with professionalism and respect other’s professionalism. Furthermore, each task should be properly delegated to the suitable person or team while simultaneously navigating boundaries between roles. In contemporary workplaces, everyone is a leader in their own role, they should be dependable and are expected to move forward without being commanded and micromanaged at every step.
Although freelancing might be appealing, it requires the maturity of enveloping all the usual roles found in a team. Let’s take a look at what roles can exist.
- Stakeholders: Generically meaning anyone that has a stake in the project
- Engineering manager: Responsible for delegating task to people, managing people
- Product manager: Knows everything about the project, the what of the project
- Software, project lead: Implements the project with the help of a team, knows how to do it
- Software architect: Designs, documents, analyses requirements and help the team keep on track to have a solution that stands the test of time
- Software engineer: Generic term for someone that works with software
- IT, networking infrastructure: Is responsible for physical infrastructures, usually related to network, and their configuration
- Operation team: A role that is slowly getting deprecated, they are tasked with going on premise or remotely installing, packaging, configuring, monitoring, and supporting a product and system once it is live.
- Database administration: The keeper of the data
- Site reliability Engineer: Sort of an upgrade on the operation team with the addition that they focus on scalable and reliable systems
- QA (quality assurance): Ensures the software is up to quality standards before going live by making it pass tests, sometimes in a pipeline. This can also include following live metrics
- Graphic UI/UX designer: Is in charge of graphics and images, how people will use the interface, what it will look like. They have back and forth behaviour testing and can also be interested in live metrics regarding this
- Frontend engineer: Engineers that will implement the front-facing, graphical part, of the application
- Backend engineer: Engineers that will implement the business rules of the application which the front-facing part relies on
- DevOps: A mix between a software engineer and an operation engineer, usually in an environment where continuous integration is in place
- Senior engineer: Someone that has worked long enough on projects in the company or with a certain technology to know it from head to toe
- Consultant: An expert hired by the company to give inputs and guidance in a project
- Fullstack engineer: An engineer burdened with doing frontend, backend, operation, and database management
Reading and Writing Documentation
Contrary to popular beliefs, programming isn’t about mindlessly
click-clacking on the keyboard from 9-to-5 and then clocking out,
nor are most jobs these days for that matter. A big chunk of time is
allocated to reading and understanding requirements, researching, and
then planning how to tackle problems. Consequently, getting comfy with
technical documents is a must.
It’s not uncommon to be spectator to the shocked look on new developers faces as they open big documentation pages for the first time. The sight of such gargantuan documents, to a generation that hates reading and has a decreasing attention span, appears like a monster impossible to tame. However, it’s all because they approach it with the wrong mentality; they are too accustomed to having to memorize and being interrogated on anything they’ve set their eyes upon. Instead, documentation should be taken as a treasure hunt: finding, extracting, and translating value into workable solutions to problems.
Similar to how we read research papers, documentation should be read from the generic to the specific, focusing on units of understanding. It could be read from “cover to cover” but it usually isn’t. Let’s also mention that there are different types of documentation, each with a different aim: tutorials, how-to, explanations, and reference guides. Though the latter is what we generally call documentation.
We recognize the high value of documentation when working with third party systems/libraries and as such should transpire this value into our own practices. Newcomers, which should already be familiar with inline comments within code, should be introduced to writing documentation using a format such as markdown, and to use diagramming tools to illustrate things.
Junior engineers and graduates have surely heard about debugging but how
many of them have actually used a debugger or thought about debugging
software at scale. Debugging is an arduous process that requires extreme
patience and calmness, two zen-like characteristics that are only acquired
by strenuous hours spent doing it.
Debugging can be executed via simple printing, via logging (distributed or not), via tracing, or via debugging tools that let us step into the code. Debugging can be enabled by configuration, by debug releases, by compiler flags, or at the test pipeline. Debugging sessions are going to take a sizable amount of time if it’s not facilitated in some way.
Analogous with debugging, logging is something we appreciate once a project reaches a certain scale. And, as with the rest of this article, it’s not often in the awareness space of junior developers.
Logging is an art and everyone is a critic of how it’s done. It can have
levels, ranging from info to error, it can have a specific format, it
can be stored in different places (file or db), it can have a rotation
scheme, it can be integrated in the software or delegated to another
(like the operating system), and it can be distributed or centralized.
The crucial part to consider is context, logging is all about keeping the context alive and being able to track it when needed.
Working From Requirements
Unlike universities and other academic institutions, workplaces rarely give instructions on how to implement solutions. They more likely share abstract requirements and user stories. The burden is on the developer to massage their way around those requirements and create a roadmap and an estimation.
The expertise to compact the requirements into a design that can openly
be discussed and defended with coworkers during meetings is what is
needed. Although, an enterprise architect could limit the choice of
technologies, even to ones you dislike, there’s still the flexibility
of how to use them.
Depending on the situation and what is agreed upon with management, choosing a method of implementation can be done based on either: experience-based analogy, back-of-the-envelope, thought experiment, simulation, prototype, experiment, or instrumentation. Then the implementation should be weighted according to its strength and weaknesses, which should be measurable. Indeed, there’s always a trade-off between the so-called “-ilities”: availability, interoperability, modifiability, performance, security, testability, usability, etc..
In spite of all this, we should avoid analysis paralysis and work on the project while still avoiding technical debt.
Programmers should comprehend the differences between the three main
programming abstraction paradigm: procedural programming, object-oriented
programming, and functional programming. Additionally, the knowledge of
different languages, their advantages and disadvantages, is a plus. From
interpreted, to just-in-time compiling, to compiled languages.
There are also high-level paradigms that can be interesting to add to a developer’s toolbox: AOP (Aspect Oriented Programming), dynamic programming, event-driven programming, natural language programming, parallel programming, meta-programming, microsystems, etc.. There’s so many of them.
Design Patterns and Modularity
The previous section leads to this one, actualizing higher paradigms
into common patterns. These design patterns have been collected and
grouped over the years because they emerged in a similar fashion from
people facing similar issues.
Learning to take advantage of them, anticipating change and minimizing complexity, is a must for professionals. Developers should have at least, once in a while, a dive into design patterns, like the GoF, to not feel out of touch.
Best Practices, and Reviews
Each programming language, each programming environment, each programming
culture, and each company, has its own set of standards and best agreed
As craftsmen, these standards are our golden rules, they are what creates cohesion between people, from the syntax formatting style used within projects, to its file structure.
Who chooses those rules depends on where they are applied, it could be a consortium, it could be the enterprise architect, it could be a senior engineer, or it could be a project lead.
In companies, the checking for compliance with the best practices normally happens as a code review session where a peer makes sure the code to be committed is up to the standards.
Here’s a list of common things that can be checked:
- The code is correct, follows the requirements, and achieves the result the stackeholders desire
- The code is easy to read and easy to understand
- The code follows the Syntax styling
- The code has test units
- The code uses strong type
- The code has too much repetition (DRY)
- The code follows defensive programming
- The design choice is good
- The design follows SOLID principles
- The lack or not of documentation
Another way to enforce standards is to enforce them programmatically
through test units. Test units are composed of a battery of tests for
verifiability (inputs and outputs to an oracle), for respect of the
architectural requirements, for interoperability checks, for speed
monitoring, and for all the other “-ilities”.
When a code is testable it is also manageable.
Other types of tests include static tests, end-to-end tests, black-white box tests, integration tests, regression tests, acceptance tests, etc..
Business is nothing without its data and by extension, its data
storage. Therefore, making the right decision on which technology to
use is crucial as data migrations are expensive and risky. To make a
decision we need basic understanding of the general characteristics
between storages and map them to our requirements.
Are we storing temporary information or long term ones, does caching suffice. Are we going for Atomicity, Consistency, Isolation, and Durability (ACID) or Basic Availability, Soft state, and Eventual consistency (BASE)? Are we going to use an Object Relational Model (ORM) to abstract access to the storage or are we going to write the queries ourselves. Are we going to have redundancy for the storage, and how is the syncing going to be done. Are we choosing a distributed data storage. Are we going for sql, nosql, or something else.
It’s rare to encounter softwares that aren’t flexible and don’t allow their users to configure them. A certain ease with textual configuration formats is required from professional software engineers. At a glance, it should be obvious if we are dealing with YAML, XML, JSON, or a custom Domain Specific Language (DSL). Everyone develops their own preferences.
Since the 60s, people have been sharing code amongst themselves for
convenience purposes. Initially, it was done manually by FTP and email but
nowadays we share libraries through the package manager of the language
we are using, or the operating system global package manager.
Seeking dependencies, managing them, and choosing them isn’t always straight forward. Does it do what we want, does it come with documentation, is it open source, and is it maintained and will it be maintained in the future. Those are questions that should be asked before committing to a third party library.
Content Versioning System
How can a team work on the same codebase without conflict and how
to know who did what at which time, this is what a content versioning
system resolves. Well, that’s in theory. In reality it’s not sufficient,
we also need to follow a development methodology to avoid messiness.
Git and svn are the most used CVS, if someone isn’t acquainted with them they’ll soon be, willingly or not. If someone hasn’t encountered the meaning of “merging”, they’ll soon be.
To make this less painful, shared code should be compartmentalized into separate tasks, committed to the versioning system with a meaningful traceable history.
CVS is to share code, the rest: the conversation, assignment of tasks,
bug reports, and the workflow, are done on collaboration and workflow tools
such as Github, Gitlab, Bitbucket, Fogbuzz, Jira, Trello, etc.. They all
have different usages but have in common the collaboration, communication,
and transparency components. So what’s a development methodology.
Software development methodologies are management techniques to split work into distinct phases, it’s also called a software development lifecycle (SDLC) as it tracks the software life/features/releases.
It’s trendy to talk about methodologies due to the fact they are central to the proper building rhythm of software. They’re part of the daily routine of engineers. Agile methodologies, such as scrum, XP, lean, are getting traction with their fast pace and are superseding the bad reputation of waterfall methodologies. As much as it’s tempting to give advices to newcomers regarding this topic, there’s nothing that beats hands-on experience and making ones own judgement about particular methodologies. Watching videos and arguments should give an idea of the situation.
Continuous Development, Pipelines, Deployment, Fast Release Cycles
At some point in time the software needs to go live, it needs to be
shipped to the customer. To accomplish this we could delegate it to the
QA team, then the operation team if everything is in order. That’s a
manual pipeline to reach production.
What we call continuous development is when we automate as much as possible of the deployment phase; packaging automatically after code checkout, testing automatically, and releasing online automatically. Jenkins, Spinnaker, Travis CI, and GitLab CI, are popular names of continuous integration tools.
Last but not least, enters the maintenance of software. Maintenance can be split into two parts, one is about monitoring the health of a running system and the other about upgrading legacy systems. They don’t have the same meaning but both are regarded as maintenance.
Monitoring is where the importance of logging, instrumentation, and reporting come into play. Some requirements and bugs can only be discovered in production and so we need a way to record what happens in our application and keep monitoring it for anomalies. A reliable system would send alarms to a response team in case of failure.
As for upgrading and working with legacy code, it isn’t instinctive at
all. Things that may feel simple on the surface may turn out complex
and we’ll keep wondering where the complexity emanates from. To keep
our sanity we have to remember Chesterton’s Fence principle, that maybe
there’s a reason for the complexity.
This takes us back to our first section about asking questions. We may need to ask seniors clarifications to be able to understand the whole system, which may not look impressive from the outside, but that somehow required the complexity it currently has… And perhaps our questions will remain unanswered.
In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.”
That’s it, so much content written while still not diving into any topic.
I’m aware that I’ve presented an idyllic scenario that might not be applicable to your current job. However, that doesn’t remove anything from their weight or the ability of newcomers to learn and deliberately practice them on their own time.
My hope is that having this out in the wild will act as a mini-guide to new software engineers that want to get started in their professional lives but that don’t feel like university and online courses are enough.
Have a fantastic career!
More on the topic:
- Unknown. The author of the image is not given in the book, although it might be the author of the text entry Josef Pokorný. [Public domain]
If you want to have a more in depth discussion I'm always available by email or irc.
We can discuss and argue about what you like and dislike, about new ideas to consider, opinions, etc..
If you don't feel like "having a discussion" or are intimidated by emails then you can simply say something small in the comment sections below and/or share it with your friends.