Author: savas

Below are our photos from the final session in August 2018 at UMBC:

Now we turn up the heat a little. It’s getting interesting! Don’t get frustrated, but I know it’s easier said than done. Microsoft is frustrating by design. When developing with Microsoft tools it’s as if you get paid for your patience, not for actual development work….:)

In the virtual machine you created in the earlier milestone #1, which was either a Windows 10 Pro Enterprise or Windows Server 2016 operating system, install Microsoft Internet Information Service, see https://www.iis.net/

You install it from the Server Manager or add/remove Windows features in Windows 10 Pro. You may need the CD (ISO) or an internet connection, I’m not sure.

Stage #1: Set up a website in IIS that runs on port 80. Create a file index.html with some basic text in it. Use anonymous login, so you can open the website without a user name and password. Open the inbound firewall port 80 so that the website can be opened by a browser on your ‘real’ computer or from another VM. Hint: always use the advanced firewall screen, it’s the best way to confuse yourself but that way you only need one screen to work with.

Stage #2: Set up the FTP server in IIS. Open ports 21 and a range of PASV ports, for example ports 10000-20000. Use the command “netstat -a -n” to make sure whatever port numbers you choose are not being listened to already.

Once the FTP server is running, make sure it works properly using FileZilla on the same machine. Log on and browse with FTP. Copy some files with FileZilla from and to the FTP server account. Your log in will be the same administrator login you use when logging into your VM.

Hint: due to special Microsoft design reasons, you may have to restart the FTP service in the Windows Service Manager when making drastic changes, example, changing ports.

Stage #3: Connect via FileZilla from your ‘real’ computer (the VM host) and connect to the VM’s FTP server. It should work the same way as before but has to be able to get through the firewalls. You will typically only need to adjust the VM’s firewall inbound rules. Add rules there to allow inbound TCP port 21 and the range you defined, example 10000-20000.

As in the previous exercise, write it all up preferably with a few screenshots.

Naturally we can find a library shelf full with books on software design, quality, and testing. What I’m presenting below is just a little note on the subject. I feel like preaching a little today…(^_^)

The thinking exercise last week, to envision yourself as a chocolate factory owner and as a software freelancer, had several goals behind it:

1. To get into an unknown industry an analyze their way of work, their entire business. As you go from junior developer to the next stage, what counts next is not so much your ability to write useful and functional software (that’s already assumed to be there), but to get into the details of a particular industry quickly. Math and computers have in common that they have little use on their own. You have to apply them somewhere to make them useful; hence, software developers who don’t have practical (hands-on) industry knowledge in a particular industry, will have difficulty entering it. For example, if you worked in the automotive industry as a developer, you can’t easily switch to the insurance and banking industry. They all prefer that you have already worked on their turf for a while so they don’t have to teach you their basics, which can be a 4-year study of its own. Trying to see the world from the chocolate factory owners view is not so easy when you don’t work in that business. That’s exactly the point of the exercise! In order to get hired in  a new industry you need the ability  to absorb industry details quickly, which requires a certain attitude towards new things. A useful attitude to have is “I want to know everything,”…. 🙂
2. To discover quality principles in a nutshell as they relate to software. As a great software professional you look always at new problems with a fresh perspective. You don’t think you have done this before even if you sort of did. You did some work before, you have some components you are familiar with that you could re-use, you talk to as many people as possible to get all details into your head. Whenever you think you figured it all out, you smack yourself in the face and tell yourself ‘don’t jump to unfounded conclusions…’ You double and triple-check everything. Your psychologist thought you were paranoid and obsessive compulsive, until she finally figured out you are an engineer. Before you type anything, you draw several plans on paper or a board and discuss your ideas with others and then decide to trash them. You try to be as ‘negative’ as possible with your plans in order to find something that’s missing. Something you forgot, that would break the ‘system’, the plan you have in mind. You look for it because you know it’s there but you can’t see it (remember you are paranoid after all…)
   1. Once the plan makes sense (and there usually is no such thing as a final plan),  and it covers all the bits and pieces you have gathered, every single requirement, you decide not to start. You think about tomorrow’s requirements. What is likely going to be tomorrow’s problem?
   2. If you write software for a client, it’s likely your client is not an IT guy and doesn’t know what s/he needs exactly. Also, because you are an “information pro” you need to break down and conceptualize your client’s data and processes  so they fit neatly into the system you are creating.
   3. If your system interfaces with other software you need to closely inspect those interfaces. They likely will be a source of headaches. Having worked in the business for years you have developed a feel for what bug you will likely encounter with other people’s code. You have started to anticipate the devil (who is in the details and) is always around the corner… Also you need to make predictions about how these interfaces are likely to change over time and be already prepared for those potential changes.
3. Break down each component into steps. If it fits it ships they say at the USPS. A piece of code (a function) has to fit on one screen or it’s too long. Just like the chocolate factory separated everything into small production steps, you also split everything into sub-functions. Every procedure you write receives inputs and creates outputs. In between, it does something. Every procedure needs an input barrier and an output barrier. Ideally no invalid signals enter undetected and undefined results ever come out without an error signal of some sort. Everything that occurs inside each procedure is logged in a log file (or use related mechanism).  Every single operation needs to be checked, even, for example, a + b, because it could result in an overflow. When you split a large process into small pieces, think about clarity and above all “code reuse”. Number #3 is the holy grail of software development. Early adopters of this methodology became productive faster and succeeded at every software project.
4. Only a small percentage of software ‘gurus’ out there are doing all of the above religiously, and they happen to be the most productive. In every company, large or small, there are a handful of these guys effectively running the entire shop. It is the ‘secret’ sauce to a 10-fold productivity and more. When you read about ‘star programmers’ and that they work like 10 or 20 developers combined (and work less than 8h/day), they do pretty much what is outlined above. It’s not a phenomenal talent, it’s just disciplined engineering. Naturally foreseeing all future problems and making code super-robust are ideals, but there is little doubt that the closer you get to such a design, the more productive and effective you become.  A lot of the productivity originates from having prepared, just like a chef in the kitchen prepares his ingredients ahead of time, reliable and immediately reusable building blocks. It’s because the remainder of all software people are spending 95% of their time ‘debugging’, only to find out their spaghetti solution isn’t working and it’s definitely not solving the problem at hand. What happens then in a typical project is that more people are added to a late project and surprisingly it is then delayed even more because even more people are now busy looking for errors instead of developing more code. Or, they finally solve the problem at hand after weeks of debugging but need to trash all the code tomorrow because of that “one little new thing” that management will ask for tomorrow and that can’t be accommodated without dramatic changes to the source code.

When you compare the two, they are like million years apart and totally different but quality principles can be applied to the software freelancer’s work process as well.
Note that the software freelancer doesn’t receive a spec sheet, “do this”. Neither did anyone go to a chocolate factory and told them “make it like this”. They have to figure out what to do and how to do it and why it’s ‘best’ done the way it’s done (or why it’s not good it’s currently done rather). And they could be totally wrong, and sometimes are.
The steps required to figure out the solution by communicating continuously with the end user, to build whatever is needed, and to manage and improve it over time, as well as to reconsider continuously whether your solution is still ‘on track’ with customer demands, is all part of a quality system.

In order to be successful in the long-term, whether you are a software freelancer, engineer, restaurant owner, or chocolate factory owner, you need to know how to set up and maintain a quality system. Even as an employee you can use quality principles to improve your work and work environment, you don’t have to be the owner.
Note that high quality doesn’t have to mean the chocolate has some magic ingredient that makes it taste ‘better’. To me McDonald’s is high quality: the food tastes like **** but it’s always the exact same taste in the same setting. The bathrooms are rather clean and the fries edible. So basically you know what you will get at McDonald’s with high certainty, whether you like it much or not doesn’t matter.  Being able to deliver consistently whatever you deliver is what sets McDonald’s a million years ahead compared to most non-fast-food restaurants.

A software freelancer doesn’t have a job or a boss. She works unsupervised on her own, using her own equipment and follows her own life and work philosophies.

In general a successful software freelancer works like this:

1. has to find or be in constant contact with clients
2. discuss client’s processes, needs, and long-term goals
3. help client’s discover their needs
4. identify problems
5. come up with solution suggestions
6. write up an agreement describing how these solutions are to be provided
7. She then works on parts of the solution (design, develop, test with real-world data and conditions)
8. while at the same time she meets with the client periodically to discuss how the work-in-progress provides the solution the client wants and needs
9. once completed the client accepts the solution or milestones thereof, and pays
10. roll-out occurs at the client site and she provides training to client’s staff
11. as the software solution is being used for the first time in production, client will ask for improvements (new features, changes, bug fixes)
12. the process then usually proceeds with item #2 for larger clients and solutions. Hence the term software development, it’s never really ‘done’.

 

Chocolate Bars Made in a Factory

Chocolate bars made in a factory are, who would have thought, made in a factory! Not uniquely, artfully, and spontaneously created by a skilled chocolatier to your custom specifications, but made by a hundred or more machines sitting in a warehouse. As we analyze the chocolate factory, contrast it to how a chocolatier perform similar steps by hand instead of using machines, and what impact that may have on the end product.

Think, for example, of a Snickers bar.

Have you ever noticed that these chocolate bars are always produced exactly the same, even after decades? Same weight, same taste, same packaging, same look, and same odor. The packaging can be opened by a two year old (is that accidental?), it has a long shelf live, and resists a rather wide temperature range without losing its main qualities. It’s easy to hold even when you are driving or working, it’s rather filling, it doesn’t leave strong odors in your mouth after you eat it (well, sort of), and for an adult it’s the right portion size. It’s not too small and not too big in size (not in sugar content, however). You will find that most children will readily want to eat one even if not hungry, i.e. it has several addictive produced ingredients that are engineered to ‘hit the spot’ quickly. After eating it you get an enormous sugar boost, unless, of course, you are already a junk food addict, in which case you have to eat it several times daily in order to avoid withdrawal symptoms. (^_^) All these product characteristics were carefully engineered.

Hence, not only the product is the same, but the food experience for the user is the same over time. The consumption is typically repeated, not a once-off transaction. Understanding that food experience must be consistent and being able to deliver such a consistent experience is what made McDonald’s the mother of all franchises.

Can you make your own Snickers, one that is exactly like one from the store? Of course not. Why exactly not?

Could you convince the Snickers makers to significantly modify their product just because you wanted it, for example, to add a piece of broccoli in the middle of your Snickers? I bet not, even if you were really rich and willing to send them all your money, they won’t be able to do it as their setup is not meant to allow for customization.

A typical Snickers factory probably produces thousands of pieces per minute, all exactly the same, day in day out. How exactly is that done? How are costs being reduced over time without affecting the quality of the product? How can product counts be increased without increasing costs and without affecting the product quality?

High production throughput and consistent quality in the food business is usually achieved through automation and continuous, careful analysis. Food production lines, similar to the assembly lines in metal industries, carry various ingredients and prepared product components step by step through dozens if not hundreds of manufacturing steps.

Each of these steps, such as preparing peanuts, can be subdivided into more steps. Each step has the power of breaking the entire production chain. Hence, whatever leaves each step must be checked carefully. As the materials move from one step to another, and certain paths are consolidated, incoming materials, their quantities and qualities must also be checked. You can’t have the chocolate mixture too hot or too cold, or too much or too little of it, not too sweet, not too bitter, etc. There are a certain number of peanut halves on the Snickers, each fitting into a specific size and weight window. The consistency of the caramel filling is also dependent on various ingredients and so forth. When the time comes to apply the caramel filling, that particular production step process has to receive specific input materials (caramel mixture, peanuts, etc) to function properly. After the machine applies the caramel topping, it needs to make sure it doesn’t apply too much or too little, and the product has to remain in a certain temperature range while all of this is done.

The factory is a collection of hundreds of tiny production steps. Some are placed in sequence, some sequences run in parallel independently for a while until they are merged later into another production branch. Each step receives materials and emits materials that were worked on. Each step takes a certain amount of time and has to stick to a certain time window for the entire process to work smoothly. The materials entering and leaving must be very tightly matching the specification. A group of people had to think long and hard to set up the entire factory, each individual step, to buy and build the machines that perform these steps, to define the input and outputs, to define the permitted variations of inputs and outputs. But setting things up was just the beginning.

Since each production step receives inputs, measures them, processes them, and measures the output, there will be a mountain of data within a few hours. The data collected needs to be analyzed, perhaps charted in minimums and maximums, trends, distributions, etc. With this data in hand, the factory operators can spot the areas that produce too much waste, or take too much time to complete. In order to make Snickers production cheaper, faster, and more effective, the output of each step has to be clearly defined in measurable variables. Obviously the taste, odor, stiffness, and other characteristics of the end product are all evaluated as much as the output of various chocolate fountains used in the production line.

To summarize: the factory operators had to break down the entire manufacturing process into small steps. Then define the inputs, outputs, and actual process taking place at each ‘product station’ as precisely as possible. Once all the production steps are chained up and working, the operators need to monitor the process by taking automated and manual records of absolutely everything that matters: temperature, quality, quantity, weight, etc., of each item moving on the belts. Each machine is evaluated. How long does it perform without requiring maintenance, what kind of maintenance does it need? Should we reduce the chocolate temperature or increase it? Does it have an effect on the taste, odor, or texture of the end product? All these variables need to be investigated until we know by trial-and-succeed what works and what doesn’t, and most importantly, why something works or doesn’t.

Every now and then, some clever analyst comes along and realizes, hey this is way more complex than it should be. She comes up with a different plan that eliminates perhaps an entire leg of the manufacturing process. Perhaps because now new technology is available or perhaps certain materials make certain new production methods more feasible. By digging where no one dug before, she found lots of opportunities to improve the process. By looking at measurements and seeing trends no one paid attention to, or only few people in the factory understood, she found lots of areas that needed a major overhaul.

The entire factory and its internal processes are therefore not really ‘set in stone’ forever, even if it appears to be from a distance. They may be fixed for a while but not forever. In a quality system, we don’t just check the outputs and expect them to be consistent (quality control), we also want to find the root of the problem and continuously improve the manufacturing process.

But there is much more to be improved over time. Unlike the Snickers bar, other products improve dramatically over time. Improvements or changes occur because of many different factors. Customers might get tired of the same thing and demand something different. Surprisingly Snickers didn’t suffer from that problem. Ingredient quality might change over time. Certain ingredients might become more expensive, so they are removed and replaced by something else, and that can improve or worsen the end product quality.

A true quality system will go far beyond the production site and figure out ways to better understand what a customer really needs, how to minimize the impact on the environment, how to work better with suppliers and distributors, it will think about what other products it could offer in the future, how to protect its workforce better from danger, and not just improve profits for its owners in the short term.

 

Please complete the following:

• Review outline at http://internship.backupchain.com/
• Create a free blog on blogger.com or wordpress.com. This will be your online portfolio / workbook. Please refrain from posting information publicly that would violate the NDA. The blog is meant to be your showcase and a little bit like a diary showing work in progress, how it progressed, what you learned and how you did it.
• Emphasis is on self-directed learning by discovery. The blog should document how learning took place. Place links to places where you found information, such as how-to videos, articles, software etc.
• Write a post, length: a few paragraphs,  introducing yourself
• Write a post, length: a few paragraphs,  introducing the idea of quality assurance and how it differs from quality control
• Write a post, length: a few paragraphs. Say you own a small chocolate bar factory. How can you benefit and use (hands-on examples) quality assurance  principles? What exactly would you suggest should be done to ensure high quality? (Assume there are no legal requirements)
• Write a post, length: a few paragraphs. Assume you are a freelance software ‘guru’.  You do everything, analyze, develop, test, and train end-users. How would you implement quality assurance principles with your next project?
• Compare the last two (factory vs. freelancer) in how they work. What differences matter and why?

Virtual Machines

• Read about virtualization, VMware vSphere, Hyper-V, VirtualBox and familiarize yourself with virtual machines, their pros and cons, and their setup
• Download and install one of the virtualization platforms on your PC, such as VMware Workstation, VirtualBox, or Hyper-V in Windows 10 Professional
• Create your first virtual machine and virtual network
• Download the ISO for Windows 10 and Windows Server 2016 from the Microsoft Evaluation Center https://www.microsoft.com/en-us/evalcenter/evaluate-windows-10-enterprise  and https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2016
• If you have time also try the ISO for Ubuntu or some other OSes.
• Configure and allow file sharing inside the VM and its firewall. Share out a folder and give access to everyone or specific users.
• Connect from the host computer via a UNC path to the VM’s file system. Try copying, opening, and editing files that are stored inside the VM.
  • familiarize yourself with the ‘net use’ command  at the command prompt (cmd.exe)
  • hint: after everything works, run ‘net use’ with the command prompt opened as your own user (i.e. simply run cmd.exe as-is) and open cmd.exe once again by right-clicking on cmd.exe and selecting ‘run as administrator’. Issue the command ‘net use’ again. What happened to your network connections?
• Connect from the VM to a network share on the host. I.e. share out a folder on your laptop so that the VM can access it via a UNC path
• Post your blog a new entry documenting your learning path.
  • First paragraph: state what you expect to achieve and how
  • Second section: document your learning stages. Where did you find info, downloads, how did you do it
  • Summary: your impressions, what was easy, what was difficult, overall impressions.
• Send an email with a link to your blog post to your team members and to internship@backupchain.com when complete or as needed

Important Notes

Note #1: The Chocolate Factory

Note #2: Software Freelancer

Additional notes

Notes on Software Design and Testing