Jekyll2023-06-05T21:02:27+00:00https://abdelrahmanhosny.me/feed.xmlAbdelrahman HosnyAbdelrahman Hosny is a Computer Science PhD Candidate at Brown University. He is interested in solving optimization problems using machine learning methods.
Abdelrahman Hosnyabdelrahman@brown.eduSummary of Hooked2023-01-01T00:00:00+00:002023-01-01T00:00:00+00:00https://abdelrahmanhosny.me/tech/products/hooked<h2 id="introduction">Introduction</h2>
<ul>
<li>Habits are defined as “behaviors done with little or no conscious thought”.</li>
<li>The convergence of access, data and speed is making the world a more habit-forming
place.</li>
<li>Businesses that create customer habits gain a significant competitive advantage.</li>
<li>The Hook Model describes an experience designed to connect the user’s problem to a
solution frequently enough to form a habit.</li>
<li>The Hook Model has four phases: trigger, action, variable reward, and investment.</li>
</ul>
<h2 id="the-habit-zone">The Habit Zone</h2>
<ul>
<li>For some businesses, forming habits is a critical component to success, but not every business requires habitual user engagement.</li>
<li>When successful, forming strong user habits can have several business benefits including: higher customer lifetime value (CLTV), greater pricing flexibility, supercharged growth, and a sharper competitive edge.</li>
<li>Habits cannot form outside the habit zone, where the behavior occurs with enough frequency and perceived utility.</li>
<li>Habit-forming products often start as nice-to-haves (vitamins), but once the habit is formed, they become must-haves (painkillers).</li>
<li>Habit-forming products alleviate users’ pain by relieving a pronounced itch.</li>
<li>Designing habit-forming products is a form of manipulation. Product builders would benefit from a bit of introspection before attempting to hook users to make sure they are building healthy habits, not unhealthy addictions.</li>
</ul>
<h3 id="trigger">Trigger</h3>
<ul>
<li>Triggers cue the user to take action and are the first step in the Hook Model.</li>
<li>Triggers come in two types - external and internal.
<ul>
<li>External:
<ul>
<li>Paid triggers</li>
<li>Earned triggers (PR work)</li>
<li>Relationship triggers (word of mouth)</li>
<li>Owned triggers (next-trigger on the platform right after external trigger)</li>
</ul>
</li>
<li>Internal
<ul>
<li>Whatever from within the platform that creates engagement and form habits on the long run</li>
</ul>
</li>
</ul>
</li>
<li>External triggers tell the user what to do next by placing information within the user’s
environment.</li>
<li>Internal triggers tell the user what to do next through associations stored in the user’s memory (brain).</li>
<li>Negative emotions frequently serve as internal triggers</li>
<li>To build a habit-forming product, makers need to understand which user emotions may
be tied to internal triggers and know how to leverage external triggers to drive the user to
action.</li>
<li>Ask “Why” five times until you reach an answer that accurately describe the internal
feeling of the user of the product. Remember FOMO (Fear Of Missing Out).</li>
</ul>
<h3 id="action">Action</h3>
<ul>
<li>B = MAT (Behavior = Motivation, Ability, Trigger)
<ul>
<li>Ability should be simple. Six elements of simplicity
<ul>
<li>Time - how long it takes to complete an action</li>
<li>Money - the fiscal cost of taking an action</li>
<li>Physical effort - the amount of labor involved in taking the action</li>
<li>Brain cycles - the level of mental effort and focus required to take an action</li>
<li>Social deviance - how accepted the behavior is by others</li>
<li>Non-routine - how much the action matches or disrupts the existing routines</li>
</ul>
</li>
<li>Motivation or ability. Which should you increase first? The answer is always ability.</li>
</ul>
</li>
<li>Brain Biases <a href="http://getmentalnotes.com/">http://getmentalnotes.com/</a>
<ul>
<li>The scarcity effect: the appearance of scarcity affects the perception of value (jar with 10 cookies or jar with 2 cookies; Amazon with only 9 left in stock. really?)</li>
<li>The framing effect: the mind takes shortcuts informed by our surroundings to make quick and sometimes erroneous judgements (perform music show in subway for 10 bucks, but if it is in a hall, it would cost 100 dollars; tasting the
same wine bottle but told different prices from $5 to $90)</li>
<li>The anchoring effect: people often anchor to one piece of information when
making a decision (sales discount)</li>
<li>The endowed progress effect: increase motivation as people believe that they are
nearing the goal (car wash service with a punch card of 10 slots (2 punched) or a punch card with 8 slots (zero punched); Remember LinkedIn profile completion ratio starts with something not at zero).</li>
</ul>
</li>
<li>The action is the simplest behavior in anticipation of reward.</li>
<li>For any behavior to occur, a trigger must be present at the same time as the user has
sufficient ability and motivation to take action.</li>
<li>To increase the desired behavior, ensure a clear trigger is present; next increase ability
by making the action easier to do; finally align with the right motivator.</li>
<li>Every behavior is driven by one of three core motivators:
<ul>
<li>Seeking pleasure and avoiding pain</li>
<li>Seeking hope and avoiding fear</li>
<li>Seeking social acceptance while avoiding social rejection</li>
</ul>
</li>
<li>Heuristics are cognitive shortcuts we take to make quick decisions. Product designers
can utilize many of the hundreds of heuristics to increase the likelihood of their desired action.</li>
</ul>
<h3 id="variable-reward">Variable Reward</h3>
<ul>
<li>Variable reward is the third phase of the Hook Model, and there are three types of variable rewards: the tribe, the hunt and the self</li>
<li>Rewards of the tribe is the search for social rewards fueled by connectedness with other people.</li>
<li>Rewards of the hun is the search for material resources and information (Pinterest and slot machines). Remember how ancient humans used to chase the prey until it is tired (because heat cannot get dissipated through fur), then they kill it.</li>
<li>Rewards of the self is the search for intrinsic rewards of mastery, competence and completion.</li>
<li>When our autonomy is threatened, we feel constrained by our lack of choices and often rebel against doing a new behavior. Psychologists refer to this as reactance. Maintaining a sense of user autonomy is a requirement for repeat engagement.</li>
<li>Experiences with finite variability become increasingly predictable with the lose their appeal over time. Experiences that maintain user interest by sustaining variability with use exhibit infinite variability. Remember FarmVille example with finite variability when they introduces CityVille, ..etc.</li>
<li>Variable rewards must satisfy users’ needs while leaving them wanting to reengage with the product.</li>
</ul>
<h3 id="investment">Investment</h3>
<ul>
<li>The investment phase is the fourth step in the Hook Model.</li>
<li>Unlike the action phase, which delivers immediate gratification, the investment phase
concerns the anticipation of rewards in the future.</li>
<li>Investments in a product create preferences because of our tendency to overvalue our
work, be consistent with past behaviors and avoid cognitive dissonance.</li>
<li>Investment comes after the variable reward phase, when users are primed to reciprocate.</li>
<li>Investments increase the likelihood of users returning by improving the service the more
it is used. They enable the accrual of stored value in the form of:
<ul>
<li>Content</li>
<li>Data</li>
<li>Followers</li>
<li>Reputation</li>
<li>Skills gained</li>
</ul>
</li>
<li>Investments increase the likelihood of users passing through the Hook again by loading the next trigger to start the cycle all over again.</li>
</ul>
<h2 id="what-are-you-going-to-do-with-this">What are you going to do with this?</h2>
<ul>
<li>Ask these five questions:
<ul>
<li>What do users really want? What pain is your product relieving? (Internal trigger)</li>
<li>What brings users to your service? (External trigger)</li>
<li>What is the simplest action users take in anticipation of reward, and how can you simplify your product to make this action easier? (Action)</li>
<li>Are users fulfilled by the reward, yet left wanting more? (Variable reward)</li>
<li>What “bit of work” do users invest in your product? Does it load the next trigger
and store value to improve the product with use? (Investment)</li>
</ul>
</li>
<li>We are all in the business of persuasion.</li>
<li>To help you, as a designer of habit-forming technology, assess the morality behind how you manipulate users. It is helpful to determine which of the four categories your work fits into.
<ul>
<li>The facilitator (you use - improve people’s lives)</li>
<li>The peddler (you don’t use - improve people’s lives)</li>
<li>The entertainer (you use - don’t improve people’s lives)</li>
<li>The dealer (you don’t use - don’t improve people’s lives)</li>
</ul>
</li>
</ul>Abdelrahman Hosnyabdelrahman@brown.eduIntroduction Habits are defined as “behaviors done with little or no conscious thought”. The convergence of access, data and speed is making the world a more habit-forming place. Businesses that create customer habits gain a significant competitive advantage. The Hook Model describes an experience designed to connect the user’s problem to a solution frequently enough to form a habit. The Hook Model has four phases: trigger, action, variable reward, and investment. The Habit Zone For some businesses, forming habits is a critical component to success, but not every business requires habitual user engagement. When successful, forming strong user habits can have several business benefits including: higher customer lifetime value (CLTV), greater pricing flexibility, supercharged growth, and a sharper competitive edge. Habits cannot form outside the habit zone, where the behavior occurs with enough frequency and perceived utility. Habit-forming products often start as nice-to-haves (vitamins), but once the habit is formed, they become must-haves (painkillers). Habit-forming products alleviate users’ pain by relieving a pronounced itch. Designing habit-forming products is a form of manipulation. Product builders would benefit from a bit of introspection before attempting to hook users to make sure they are building healthy habits, not unhealthy addictions. Trigger Triggers cue the user to take action and are the first step in the Hook Model. Triggers come in two types - external and internal. External: Paid triggers Earned triggers (PR work) Relationship triggers (word of mouth) Owned triggers (next-trigger on the platform right after external trigger) Internal Whatever from within the platform that creates engagement and form habits on the long run External triggers tell the user what to do next by placing information within the user’s environment. Internal triggers tell the user what to do next through associations stored in the user’s memory (brain). Negative emotions frequently serve as internal triggers To build a habit-forming product, makers need to understand which user emotions may be tied to internal triggers and know how to leverage external triggers to drive the user to action. Ask “Why” five times until you reach an answer that accurately describe the internal feeling of the user of the product. Remember FOMO (Fear Of Missing Out). Action B = MAT (Behavior = Motivation, Ability, Trigger) Ability should be simple. Six elements of simplicity Time - how long it takes to complete an action Money - the fiscal cost of taking an action Physical effort - the amount of labor involved in taking the action Brain cycles - the level of mental effort and focus required to take an action Social deviance - how accepted the behavior is by others Non-routine - how much the action matches or disrupts the existing routines Motivation or ability. Which should you increase first? The answer is always ability. Brain Biases http://getmentalnotes.com/ The scarcity effect: the appearance of scarcity affects the perception of value (jar with 10 cookies or jar with 2 cookies; Amazon with only 9 left in stock. really?) The framing effect: the mind takes shortcuts informed by our surroundings to make quick and sometimes erroneous judgements (perform music show in subway for 10 bucks, but if it is in a hall, it would cost 100 dollars; tasting the same wine bottle but told different prices from $5 to $90) The anchoring effect: people often anchor to one piece of information when making a decision (sales discount) The endowed progress effect: increase motivation as people believe that they are nearing the goal (car wash service with a punch card of 10 slots (2 punched) or a punch card with 8 slots (zero punched); Remember LinkedIn profile completion ratio starts with something not at zero). The action is the simplest behavior in anticipation of reward. For any behavior to occur, a trigger must be present at the same time as the user has sufficient ability and motivation to take action. To increase the desired behavior, ensure a clear trigger is present; next increase ability by making the action easier to do; finally align with the right motivator. Every behavior is driven by one of three core motivators: Seeking pleasure and avoiding pain Seeking hope and avoiding fear Seeking social acceptance while avoiding social rejection Heuristics are cognitive shortcuts we take to make quick decisions. Product designers can utilize many of the hundreds of heuristics to increase the likelihood of their desired action. Variable Reward Variable reward is the third phase of the Hook Model, and there are three types of variable rewards: the tribe, the hunt and the self Rewards of the tribe is the search for social rewards fueled by connectedness with other people. Rewards of the hun is the search for material resources and information (Pinterest and slot machines). Remember how ancient humans used to chase the prey until it is tired (because heat cannot get dissipated through fur), then they kill it. Rewards of the self is the search for intrinsic rewards of mastery, competence and completion. When our autonomy is threatened, we feel constrained by our lack of choices and often rebel against doing a new behavior. Psychologists refer to this as reactance. Maintaining a sense of user autonomy is a requirement for repeat engagement. Experiences with finite variability become increasingly predictable with the lose their appeal over time. Experiences that maintain user interest by sustaining variability with use exhibit infinite variability. Remember FarmVille example with finite variability when they introduces CityVille, ..etc. Variable rewards must satisfy users’ needs while leaving them wanting to reengage with the product. Investment The investment phase is the fourth step in the Hook Model. Unlike the action phase, which delivers immediate gratification, the investment phase concerns the anticipation of rewards in the future. Investments in a product create preferences because of our tendency to overvalue our work, be consistent with past behaviors and avoid cognitive dissonance. Investment comes after the variable reward phase, when users are primed to reciprocate. Investments increase the likelihood of users returning by improving the service the more it is used. They enable the accrual of stored value in the form of: Content Data Followers Reputation Skills gained Investments increase the likelihood of users passing through the Hook again by loading the next trigger to start the cycle all over again. What are you going to do with this? Ask these five questions: What do users really want? What pain is your product relieving? (Internal trigger) What brings users to your service? (External trigger) What is the simplest action users take in anticipation of reward, and how can you simplify your product to make this action easier? (Action) Are users fulfilled by the reward, yet left wanting more? (Variable reward) What “bit of work” do users invest in your product? Does it load the next trigger and store value to improve the product with use? (Investment) We are all in the business of persuasion. To help you, as a designer of habit-forming technology, assess the morality behind how you manipulate users. It is helpful to determine which of the four categories your work fits into. The facilitator (you use - improve people’s lives) The peddler (you don’t use - improve people’s lives) The entertainer (you use - don’t improve people’s lives) The dealer (you don’t use - don’t improve people’s lives)Getting Started with OpenROAD (Part 1)2019-12-06T00:00:00+00:002019-12-06T00:00:00+00:00https://abdelrahmanhosny.me/tech/eda/getting-started-with-openroad-1<p>In digital design, a circuit is described in a hardware description language (e.g. Verilog) and the goal is to manufacture it.
To get the actual layout of the circuit that is manufacturable, it needs to pass through a number of steps before handing it over to a fabrication lab.
In this post, I briefly give an overview of the steps and show how to perform them using open-source tools in OpenROAD.</p>
<p>Website: <a href="https://theopenroadproject.org/">https://theopenroadproject.org/</a></p>
<p>GitHub: <a href="https://github.com/The-OpenROAD-Project">https://github.com/The-OpenROAD-Project</a></p>
<p>Twitter: <a href="https://twitter.com/OpenROAD_EDA">https://twitter.com/OpenROAD_EDA</a></p>
<!--more-->
<h1 id="before-you-read">Before you read</h1>
<p>This post assumes that you have some background on hardware design.
No programming knowledge is required to use the tools.
Physical design knowledge is a plus, but not necessary.
When appropriate, I give some links to Wikipedia articles to refer to.</p>
<p>This post assumes that you have some background on hardware design.
No programming knowledge is required to use the tools.
Physical design knowledge is a plus, but not necessary.
When appropriate, I give some links to Wikipedia articles to refer to.</p>
<blockquote>
<p>Disclaimer: I’m part of OpenROAD project team. Thanks to all my colleagues for putting together this effort.</p>
</blockquote>
<blockquote>
<p>NOTE: If you are reading this post after July 2020, the chances are that this post is obsolete and there is another up-to-date writing on OpenROAD. Please, see other posts on this blog or refer to the OpenROAD website.</p>
</blockquote>
<h1 id="so-you-have-a-circuit-design-">So, you have a circuit design ..</h1>
<p>.. and you want to get it manufactured. For example, the below is a desciption of a multiplier circtuit in Verilog.</p>
<div class="language-verilog highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="n">multiplier</span><span class="p">(</span><span class="n">clk</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">);</span>
<span class="kt">input</span> <span class="n">clk</span><span class="p">;</span>
<span class="kt">input</span> <span class="p">[</span><span class="mi">7</span><span class="o">:</span><span class="mi">0</span><span class="p">]</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">;</span>
<span class="kt">output</span> <span class="kt">reg</span> <span class="p">[</span><span class="mi">15</span><span class="o">:</span><span class="mi">0</span><span class="p">]</span> <span class="n">c</span><span class="p">;</span>
<span class="k">always</span> <span class="o">@</span><span class="p">(</span><span class="kt">posedge</span> <span class="n">clk</span><span class="p">)</span>
<span class="n">c</span> <span class="o"><=</span> <span class="n">a</span><span class="o">*</span><span class="n">b</span><span class="p">;</span>
<span class="k">endmodule</span>
</code></pre></div></div>
<p>This is a simple circuit that takes two inputs, <code class="language-plaintext highlighter-rouge">a</code> and <code class="language-plaintext highlighter-rouge">b</code>, 8 bits each and multiplies them to produce the result in a 16-bit register <code class="language-plaintext highlighter-rouge">c</code>.
We call this abstraction <a href="https://en.wikipedia.org/wiki/Register-transfer_level">Register-Transfer Level (RTL)</a> because it models the functionality of a synchronous digital circuit in terms of the flow of digital signals (data) between hardware registers, and the logical operations performed on those signals. Although the multiplier circuit is less than 10 lines of code, the gate representation of the circuit is not simple. Gate representation are the actual <a href="https://en.wikipedia.org/wiki/Logic_gate">logic gates</a> that implement the functionality of the circuit, such as <code class="language-plaintext highlighter-rouge">AND</code>, <code class="language-plaintext highlighter-rouge">OR</code> and <code class="language-plaintext highlighter-rouge">NOT</code>.</p>
<h1 id="logic-synthesis">Logic Synthesis</h1>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input: RTL (design.v)
Output: Gate-level (netlist.v)
</code></pre></div></div>
<p><a href="https://en.wikipedia.org/wiki/Logic_synthesis">Logic synthesis</a> is the first step in this flow, which takes as input the RTL (basically the Verilog design) and outputs a <a href="https://en.wikipedia.org/wiki/Netlist">gate-level netlist</a> that describes the connectivity of the logic gates.
The logic synthesis tool takes care of this transformation. One such open-source tool is <a href="https://github.com/YosysHQ/yosys">Yosys</a>.</p>
<h1 id="physical-design">Physical Design</h1>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input: Gate-level netlist (netlist.v)
Output: Graphic Database System format (layout.gds)
</code></pre></div></div>
<p>Logic circuits are manufactured using <a href="https://en.wikipedia.org/wiki/Transistor">transistors</a>. A combination of transistors arranged in a specific layout perform the functionality of a logic gate. The goal of the <a href="https://en.wikipedia.org/wiki/Physical_design_(electronics)">Physical Design</a> step is to obtain the geometric representations of the shapes (of transistors and wire interconnects), that when manufactured in the corresponding layers of material, will produce the required functionality of the circuit (e.g. multiply two numbers). This layout description is defined in a binary format called <a href="https://en.wikipedia.org/wiki/GDSII">GDSII</a>.</p>
<p><strong>But how can we go from Netlist to GDS?</strong></p>
<p>There are several steps that are necessary to complete in order to get a manufacturable circuit. Let’s give a high-level overview on them below.</p>
<h2 id="floorplanning">Floorplanning</h2>
<p><a href="https://en.wikipedia.org/wiki/Floorplan_(microelectronics)">Floorplanning</a> produces a tentative representation of where the major blocks of the circuits are placed. In particular, we want to specify the following:</p>
<ul>
<li>What is the area of the chip? And what are the dimensions within which, the gates and wires will be placed?</li>
<li>Where are the input and output pins? For example, in the multiplier above, we have 8 pins for input <code class="language-plaintext highlighter-rouge">a</code>, 8 pins for input <code class="language-plaintext highlighter-rouge">b</code>, 1 pin for the <code class="language-plaintext highlighter-rouge">clk</code> input and 16 pins for the output <code class="language-plaintext highlighter-rouge">c</code>. Now, where on the perimeter of the chip should these pins be placed?</li>
<li>How are we connecting power/ground to each of the individual gates? This is known as the <a href="https://en.wikipedia.org/wiki/Power_network_design_(IC)">Power Delivery Network (PDN)</a>.</li>
<li>Are we placing macros or IP blocks somewhere on the chip? Where are they located? Should they be connected to any IO pins?</li>
</ul>
<p>OpenROAD has tools that automates the above process. See: <a href="https://github.com/The-OpenROAD-Project/ioPlacer">ioPlacer</a>, <a href="https://github.com/The-OpenROAD-Project/TritonMacroPlace">MacroPlacer</a>, <a href="https://github.com/The-OpenROAD-Project/pdn">pdn</a> and <a href="https://github.com/The-OpenROAD-Project/tapcell">tapcell</a>.</p>
<h2 id="placement">Placement</h2>
<p><a href="https://en.wikipedia.org/wiki/Placement_(electronic_design_automation)">Placement</a> is the step where we identify the exact locations of the different circuit components (i.e. cells). The exact location is given by the <code class="language-plaintext highlighter-rouge">(x,y)</code> coordinates of the cell on the chip described in the floorplanning step.</p>
<p>In modern placement tools, this step is divided into global placement and detailed placement. Global placement distributes all the cells to appropriate locations in the global scale with some overlaps (which violates the design rules). Detailed placement shifts each instance to nearby legal location with very moderate layout change so that the layout is correct and can be manufactured.</p>
<p>OpenROAD has tools that automates the placement process. See <a href="https://github.com/The-OpenROAD-Project/RePlAce">RePlAce</a> and <a href="https://github.com/The-OpenROAD-Project/OpenDP">OpenDP</a>.</p>
<h2 id="clock-tree-synthesis">Clock Tree Synthesis</h2>
<p>When circuits have sequential elements that require synchronization, such as <a href="https://en.wikipedia.org/wiki/Flip-flop_(electronics)">flip-flops</a>, <a href="https://vlsibasic.blogspot.com/2014/01/clock-tree-synthesis.html?m=1">Clock Tree Synthesis (CTS)</a> is the process that distributes the <code class="language-plaintext highlighter-rouge">clk</code> signal evenly to all sequential elements of the circuit. The goal of the CTS step is to reduce clock <a href="https://en.wikipedia.org/wiki/Clock_skew">skew</a> and <a href="https://en.wikipedia.org/wiki/Clock_synchronization">latency</a>.</p>
<p>OpenROAD uses <a href="https://github.com/The-OpenROAD-Project/TritonCTS">TritonCTS</a> to generate the clock tree.</p>
<h2 id="routing">Routing</h2>
<p>Now that we know the location of every element in the circuit (also referred to as cells), we want to describe how these elements are wired. Wires are defined as an exact geometric path on the metal layers that are used for fabrication. This is where the <a href="https://en.wikipedia.org/wiki/Routing_(electronic_design_automation)">Routing</a> step comes into place. It builds upon the placement step in order to have a fully connected circuit that functions properly.</p>
<p>The challenge in the placement/CTS/routing steps is that all of the algorithms used have to obey some technology rules (i.e. constraints) while describing the layout of the circuit. These are also referred to as <strong>Design Rules</strong>, and they are usually given by the technology foundry that are targeted for manufacturing.</p>
<p>OpenROAD uses <a href="https://github.com/The-OpenROAD-Project/FastRoute">FastRoute</a> and <a href="https://github.com/The-OpenROAD-Project/TritonRoute">TritonRoute</a> as the global and detailed routing tools respectively.</p>
<h1 id="getting-started-with-openroad">Getting Started with OpenROAD</h1>
<p>Now, that we understand the main steps involved in physical design, we are ready to see how they are performed in OpenROAD.</p>
<p>OpenROAD targets a no-human-in-loop (NHIL) design.
No humans means that tools must adapt and self-tune, and never get stuck within the layout generation process.
That’s why using OpenROAD tools doesn’t require the knowledge of physical design.</p>
<h2 id="what-you-need-to-know">What you need to know</h2>
<p><strong>The OpenROAD v1.0 tool, to be released in July 2020, will be capable of push-button, DRC-clean RTL-to-GDS layout generation in a commercial FinFET process node.</strong></p>
<p>The tool is currently visible at: <a href="https://github.com/The-OpenROAD-Project/OpenROAD">https://github.com/The-OpenROAD-Project/OpenROAD</a></p>
<p>In its v1.0 form, it will be integrated on an incremental substrate provided by the <a href="https://github.com/The-OpenROAD-Project/OpenDB">OpenDB</a> database and the <a href="https://github.com/The-OpenROAD-Project/OpenSTA">OpenSTA</a> static timing engine.
It will also offer users and developers Tcl/Python scripting interfaces, and support SoC designs.
At the same time, the functionality of OpenROAD v1.0 will be highly limited relative to that of commercial EDA tools that IC designers are familiar with.
Further, the development resources of the OpenROAD project are being largely focused on support of a ~July 2020 SoC tapeout in a commercial FinFET node.</p>
<p>Please, refer to <a href="https://vlsicad.ucsd.edu/NEWS19/OpenROAD%20RTL-to-GDS%20v1.0%20Expectations.pdf">OpenROAD v1.0 Expectations</a> for a detailed description on the capabilities of the tools used.</p>
<h2 id="installing-the-tools">Installing the tools</h2>
<p>Continue to <a href="/tech/eda/2019-12-06-getting-started-with-openroad-2/">Part 2</a> ..</p>
<p>Please, leave questions in the comments below and I will respond as soon as possible ..</p>Abdelrahman Hosnyabdelrahman@brown.eduIn digital design, a circuit is described in a hardware description language (e.g. Verilog) and the goal is to manufacture it. To get the actual layout of the circuit that is manufacturable, it needs to pass through a number of steps before handing it over to a fabrication lab. In this post, I briefly give an overview of the steps and show how to perform them using open-source tools in OpenROAD. Website: https://theopenroadproject.org/ GitHub: https://github.com/The-OpenROAD-Project Twitter: https://twitter.com/OpenROAD_EDAGetting Started with OpenROAD (Part 2)2019-12-06T00:00:00+00:002019-12-06T00:00:00+00:00https://abdelrahmanhosny.me/tech/eda/getting-started-with-openroad-2<p>In the <a href="/tech/eda/2019-12-06-getting-started-with-openroad-1/">previous post</a>, I have given a brief introduction on physical design and the steps needed to manufacture a hardware circuit. In this post, I will showcase the use of OpenROAD tools to perform the tasks discussed in the physical design process.</p>
<h1 id="prerequisites">Prerequisites</h1>
<h2 id="machine-specs">Machine specs</h2>
<p>To follow up with this tutorial, we recommend the use of a machine with 4GB+ of RAM to build and run the tools. However, if you are using big designs, more memory is required for the tools to perform their computations in a timely manner.</p>
<!--more-->
<blockquote>
<p>Disclaimer: I’m part of OpenROAD project team. Thanks to all my colleagues for putting together this effort.</p>
</blockquote>
<blockquote>
<p>NOTE: If you are reading this post after July 2020, the chances are that this post is obsolete and there is another up-to-date writing on OpenROAD. Please, see other posts on this blog or refer to the OpenROAD website.</p>
</blockquote>
<h2 id="operating-system">Operating system</h2>
<p>While the tools can theoretically be built on Linux, Windows and Mac, they have been tested only on CentOS 7. If you have a MacOS or a Windows machine, please, <a href="https://docs.docker.com/v17.09/engine/installation/">install Docker</a> to follow up with the tutorial.</p>
<h2 id="install-the-required-packages">Install the required packages</h2>
<h3 id="bare-metal">Bare-metal</h3>
<ol>
<li>Install essential build tools
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> yum group <span class="nb">install</span> <span class="nt">-y</span> <span class="s2">"Development Tools"</span>
yum <span class="nt">-y</span> <span class="nb">install </span>centos-release-scl
yum <span class="nt">-y</span> <span class="nb">install </span>devtoolset-8 devtoolset-8-libatomic-devel
</code></pre></div> </div>
</li>
<li>Install CMake
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> wget https://cmake.org/files/v3.14/cmake-3.14.0-Linux-x86_64.sh
<span class="nb">chmod</span> +x cmake-3.14.0-Linux-x86_64.sh
./cmake-3.14.0-Linux-x86_64.sh <span class="nt">--skip-license</span> <span class="nt">--prefix</span><span class="o">=</span>/usr/local
</code></pre></div> </div>
</li>
<li>Install development dependencies
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> yum <span class="nb">install</span> <span class="nt">-y</span> wget git
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum <span class="nb">install</span> <span class="nt">-y</span> epel-release-latest-7.noarch.rpm
yum <span class="nb">install</span> <span class="nt">-y</span> tcl-devel tcl tk libstdc++ tk-devel boost-devel
</code></pre></div> </div>
</li>
<li>Install Python development dependencies
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> yum <span class="nb">install</span> <span class="nt">-y</span> https://centos7.iuscommunity.org/ius-release.rpm
yum update <span class="nt">-y</span>
yum <span class="nb">install</span> <span class="nt">-y</span> python36u python36u-libs python36u-devel python36u-pip
</code></pre></div> </div>
</li>
</ol>
<h3 id="docker">Docker</h3>
<p>A docker image with all required packages pre-installed is available at <a href="https://hub.docker.com/repository/docker/openroad/openroad">openroad/openroad:base</a>. Use the following command to pull the image from Docker Hub.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker pull openroad/openroad:base
</code></pre></div></div>
<h2 id="build-openroad">Build OpenROAD</h2>
<ol>
<li>Clone the repository
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> git clone <span class="nt">--recursive</span> https://github.com/The-OpenROAD-Project/OpenROAD.git
</code></pre></div> </div>
</li>
<li>Change to repo directory
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nb">cd </span>OpenROAD
</code></pre></div> </div>
<p>If you are using Docker, then run the following command and continue working inside the container for the rest of the tutorial.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> docker run <span class="nt">-it</span> <span class="nt">-v</span> <span class="si">$(</span><span class="nb">pwd</span><span class="si">)</span>:/OpenROAD openroad/openroad:base bash
<span class="nb">cd</span> /OpenROAD
</code></pre></div> </div>
</li>
<li>Build OpenROAD
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nb">mkdir </span>build
<span class="nb">cd </span>build
cmake ..
make <span class="nt">-j</span> 4
make <span class="nb">install</span>
</code></pre></div> </div>
</li>
<li>Test OpenROAD
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> openroad <span class="nt">-version</span>
</code></pre></div> </div>
<p>You should see openroad version printed out along with the hash of the build.</p>
</li>
</ol>
<h2 id="prepare-your-data">Prepare your data</h2>
<h3 id="technology-node">Technology Node</h3>
<p>Typically, a fabrication lab provides designers with the necessary files that describe supported cells along with their physical and geometric characteristics. These files are referred to as <a href="https://en.wikipedia.org/wiki/Standard_cell">Standard Cell Library</a>.</p>
<p>In this tutorial, we will use an open-source library called <a href="https://projects.si2.org/openeda.si2.org/help/group_ld.php?group=63">Nangate</a>. For your convenience, I provide direct download links:</p>
<ul>
<li><a href="/assets/eda/nangate.lib">nangate.lib</a>: the lib file includes descriptions of the standard cell functionalities, along with some of their physical properties such as timing and power.</li>
<li><a href="/assets/eda/nangate.lef">nangate.lef</a>: the lef file (or <a href="https://en.wikipedia.org/wiki/Library_Exchange_Format">library exchange format</a>) includes specification of the cell geometries and metal layers, in addition to the design rules for the target technology.</li>
</ul>
<p>The original library file can be downloaded from <a href="https://github.com/The-OpenROAD-Project/alpha-release/tree/master/flow/platforms/nangate45">here</a>.
See licence information of using this library <a href="/assets/eda/OpenCellLibraryLicenseSi2.txt">here</a>.</p>
<h3 id="design">Design</h3>
<p>There are many open-source designs that you can try out from <a href="https://github.com/lsils/benchmarks">EPFL benchmark suite</a> and <a href="https://opencores.org/">OpenCores</a>. In this tutorial, we will use a simple design that calculates the <em>greatest common divisor (GCD)</em> between two numbers. For your convenience, you can download a netlist of the design from <a href="/assets/eda/gcd_netlist.v">here</a>.</p>
<p>The original design file can be downloaded from <a href="https://github.com/The-OpenROAD-Project/alpha-release/tree/master/flow/designs/src/gcd">here</a>.</p>
<h2 id="run-openroad">Run OpenROAD</h2>
<p>After copying the above files to the <code class="language-plaintext highlighter-rouge">build</code> directory, run OpenROAD application</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>openroad
% <type-commands-here>
</code></pre></div></div>
<p><strong>1. Load LEF file</strong></p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% read_lef nangate.lef
</code></pre></div></div>
<p><strong>2. Load netlist</strong></p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% read_verilog gcd_netlist.v
% link_design gcd
</code></pre></div></div>
<p><strong>3. Save a temporary state of the database</strong></p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% write_db gcd_tutorial.db
</code></pre></div></div>
<blockquote>
<p>NOTE: At this point, the application is ready to perform the physical design operations we discussed in the <a href="/tech/eda/2019-12-06-getting-started-with-openroad-1/">previous post</a>. At any point in time, you can just load the database file (gcd_tutorial.db) and execute the commands available in the tool.</p>
</blockquote>
<p><strong>4. Floorplanning</strong>
This command initializes the chip with a utilization of 70%.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% initialize_floorplan <span class="nt">-utilization</span> 70 <span class="nt">-site</span> FreePDK45_38x28_10R_NP_162NW_34O
</code></pre></div></div>
<p>This command places IO pins around the chip boundaries.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% auto_place_pins metal8
</code></pre></div></div>
<p><strong>5. Placement</strong></p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% global_placement
% legalize_placement
</code></pre></div></div>
<p>You should see the following report printed out:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CoreArea:
0.000000 : 0.000000 - 52060.000000 : 50400.000000
DieArea:
0.000000 : 0.000000 - 52060.000000 : 50400.000000
non_group_cell_region_assign done ..
- - - - - - - - - - - - - - - - - - - - - - - -
non_group_cell_placement done ..
- - - - - - - - - - - - - - - - - - - - - - - -
==== CHECK LEGALITY ====
row_check ==>> PASS
site_check ==>> PASS
power_check ==>> PASS
edge_check ==>> PASS
placed_check ==>> PASS
overlap_check ==>> PASS
-------------------- DESIGN ANALYSIS ------------------------------
total cells : 322
multi cells : 0
fixed cells : 0
total nets : 393
design area : 2623824000.0000
total f_area : 0.0000
total m_area : 1920520000.0000
design util : 73.1955
num rows : 18
row height : 2800.0000
-------------------------------------------------------------------
</code></pre></div></div>
<p><strong>6. Save results</strong></p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% write_def gcd_placed.def
</code></pre></div></div>
<p>This writes the design to a <a href="https://en.wikipedia.org/wiki/Design_Exchange_Format">Design Exchange Format</a> that can be used later (or with other open-source tools).</p>
<h3 id="whats-next">What’s next?</h3>
<p>This tutorial does not include the logic synthesis step, nor the routing step. This will be continued in Part 3.</p>
<p>Please, leave questions in the comments below and I will respond as soon as possible ..</p>Abdelrahman Hosnyabdelrahman@brown.eduIn the previous post, I have given a brief introduction on physical design and the steps needed to manufacture a hardware circuit. In this post, I will showcase the use of OpenROAD tools to perform the tasks discussed in the physical design process. Prerequisites Machine specs To follow up with this tutorial, we recommend the use of a machine with 4GB+ of RAM to build and run the tools. However, if you are using big designs, more memory is required for the tools to perform their computations in a timely manner.Installing Jenkins on Google Cloud Platform2019-09-23T00:00:00+00:002019-09-23T00:00:00+00:00https://abdelrahmanhosny.me/tech/continuous-integration-deployment/installing-jenkins-on-gcp<p>Continuous Integration (CI) proved to be of extreme importance to developer productivity. Implementing a CI pipeline sets the framework for code contributors to make sure their new additions/deletions improve the code base and don’t fall back to a non-ending journey of debugging. When it comes to implementing the pipeline, there are many cloud-based tools that can make this automation easier. For example, <a href="https://circleci.com/">CircleCI</a>, <a href="https://travis-ci.org/">Travis CI</a>, <a href="https://docs.gitlab.com/ee/ci/quick_start/">GitLab CI</a> and the newly-introduced (at the time of writing) <a href="https://github.com/features/actions">GitHub Actions</a>. However, there are some cases where a project needs to self-host its automation server, and this is where <a href="https://jenkins.io/">Jenkins</a> comes as a well-developed and well-maintained automation server.</p>
<p>In this post, I show how to correctly set up a prodcution-level master Jenkins server. This demo is implemented on <a href="https://cloud.google.com/">Google Cloud Platform</a> as it offers unlimited free (<em>forever</em> or until further notification) <a href="https://cloud.google.com/free/">Compute Engine instance</a>. It is a small instance (F-1 micro), but should be fine for small tasks. If you have builds that take longer times, or requires a large amount of memory, you should start a bigger server on any cloud provider. Also, we will use Debian GNU/Linux 9 as the OS on whichever instance you choose.</p>
<!--more-->
<h2 id="create-compute-engine-instance">Create Compute Engine Instance</h2>
<p><strong>STEP 1:</strong> Create an account at https://cloud.google.com and navigate to the <em>Compute Engine</em> section. It will ask you to create a project that resources will get assigned to. The screen you should start from in this tutorial is shown below.
<img src="/assets/img/posts/jenkins-gcp/jenkins-gcp-1.png" alt="" /></p>
<p><strong>STEP 2:</strong> Click <em>Create</em> and type in instance name. Choose a general purpose instance, and the size should be F-1 micro (which is the one available for free).</p>
<p><img src="/assets/img/posts/jenkins-gcp/jenkins-gcp-2.png" alt="" /></p>
<p><strong>STEP 3:</strong> Select Debian GNU/Linux 9 (stretch).</p>
<p><img src="/assets/img/posts/jenkins-gcp/jenkins-gcp-3.png" alt="" /></p>
<p><strong>STEP 4:</strong> Configure network access by choosing <em>allow defaul access</em>. Also, make sure to allow HTTP and HTTPS traffic.</p>
<p><img src="/assets/img/posts/jenkins-gcp/jenkins-gcp-4.png" alt="" /></p>
<p><strong>STEP 5:</strong> Click Create</p>
<p><img src="/assets/img/posts/jenkins-gcp/jenkins-gcp-5.png" alt="" /></p>
<p><strong>STEP 6:</strong> Once the VM is created, <code class="language-plaintext highlighter-rouge">ssh</code> to the machine from the browser by selecting from the menu <em>open in a browser window</em>.</p>
<p><img src="/assets/img/posts/jenkins-gcp/jenkins-gcp-6.png" alt="" /></p>
<p><strong>STEP 7:</strong> You are in!</p>
<p><img src="/assets/img/posts/jenkins-gcp/jenkins-gcp-7.png" alt="" /></p>
<h2 id="set-up-jenkins">Set up Jenkins</h2>
<p>In this part, we want to install jenkins and serve it at a given URL. We will also create an HTTPS certificate using <a href="https://letsencrypt.org/">let’s encrypt</a>.</p>
<h3 id="install-prerequisites">Install Prerequisites</h3>
<ol>
<li><code class="language-plaintext highlighter-rouge">sudo apt update</code>: updates packages</li>
<li><code class="language-plaintext highlighter-rouge">sudo apt install openjdk-8-jdk</code>: install Java 8 (a prerequisite for Jenkins)</li>
<li><code class="language-plaintext highlighter-rouge">sudo apt install apache2</code>: install Apache web server (to serve Jenkins traffic)</li>
<li><code class="language-plaintext highlighter-rouge">sudo apt install build-essential</code>: install build tools for some apache modules to work.</li>
<li><code class="language-plaintext highlighter-rouge">sudo a2enmod proxy && sudo a2enmod proxy_http && sudo a2enmod proxy_balancer && sudo a2enmod lbmethod_byrequests sudo a2enmod headers</code>: enable Apache modules for proxying.</li>
<li><code class="language-plaintext highlighter-rouge">sudo systemctl restart apache2</code>: restart Apache to enable changes in the modules</li>
</ol>
<p><strong>CHECK</strong>: Apache should now be installed and working properly. Go to the IP address of the instance in the web browser. You should see the default landing page of Apache. If not, something has gone wrong. Check Apache log files. Also, if you type in the terminal <code class="language-plaintext highlighter-rouge">sudo service apache2 status</code>, you should see <code class="language-plaintext highlighter-rouge">active (running)</code></p>
<h3 id="install-jenkins">Install Jenkins</h3>
<p>Refer to <a href="https://jenkins.io/doc/book/installing/">Jenkins documentation</a> if you are using a different OS.</p>
<ol>
<li><code class="language-plaintext highlighter-rouge">wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -</code></li>
<li><code class="language-plaintext highlighter-rouge">sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'</code></li>
<li><code class="language-plaintext highlighter-rouge">sudo apt update</code></li>
<li><code class="language-plaintext highlighter-rouge">sudo apt install jenkins</code></li>
<li><code class="language-plaintext highlighter-rouge">sudo systemctl start jenkins</code></li>
</ol>
<p><strong>CHECK:</strong> Jenkins is now installed. Check that it is running properly by typing in <code class="language-plaintext highlighter-rouge">sudo service jenkins status</code>. You should see <code class="language-plaintext highlighter-rouge">active (running)</code>. If not, check the logs.</p>
<h3 id="link-a-domain-name">Link a Domain Name</h3>
<ol>
<li><code class="language-plaintext highlighter-rouge">cd /etc/apache2/sites-available/</code></li>
<li><code class="language-plaintext highlighter-rouge">vim sudo vim 000-default.conf </code></li>
<li>
<p>Configure the VirtualHost to look as the following. Replace the domain name with the domain you plan to link. In my case, it is <code class="language-plaintext highlighter-rouge">jenkins.abdelrahmanhosny.me</code>. Also, edit <code class="language-plaintext highlighter-rouge">ServerAdmin</code> email.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <VirtualHost <span class="k">*</span>:80>
ServerName jenkins.abdelrahmanhosny.me
ProxyPreserveHost On
ProxyPass / http://localhost:8080/ nocanon
ProxyPassReverse / http://localhost:8080/
RequestHeader <span class="nb">set</span> <span class="s2">"X-Forwarded-Proto"</span> <span class="nb">expr</span><span class="o">=</span>%<span class="o">{</span>REQUEST_SCHEME<span class="o">}</span>
ProxyRequests Off
AllowEncodedSlashes NoDecode
ServerAdmin email@domain
DocumentRoot /var/www/html/jenkins
ErrorLog <span class="k">${</span><span class="nv">APACHE_LOG_DIR</span><span class="k">}</span>/error-jenkins.log
CustomLog <span class="k">${</span><span class="nv">APACHE_LOG_DIR</span><span class="k">}</span>/access-jenkins.log combined
</VirtualHost>
</code></pre></div> </div>
</li>
<li><code class="language-plaintext highlighter-rouge">sudo systemctl restart apache2</code></li>
<li><strong>IMPORTANT:</strong> Go to your domain registrar and update the <code class="language-plaintext highlighter-rouge">A</code> record of the domain to point to the IP address of your instance. Allow for some time for the DNS recrods to propagate (usually about 10 minutes).</li>
</ol>
<p><strong>CHECK:</strong> In a new browser tab, type in the domain name, it should open the defaul Jenkins page. Do <strong>NOT</strong> configure Jenkins yet. If the page is not loading, the DNS record might be unset yet. If you waited for more time, try checking <code class="language-plaintext highlighter-rouge">host <domain name></code> in your local terminal. If it shows the IP address of the machine, then some other error in the server happened. Check the Apache log.</p>
<h3 id="obtain-https-certificate">Obtain HTTPS Certificate</h3>
<ol>
<li><code class="language-plaintext highlighter-rouge">sudo add-apt-repository ppa:certbot/certbot</code></li>
<li>
<p><code class="language-plaintext highlighter-rouge">vim /etc/apt/sources.list</code> and add the following lines.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> deb http://deb.debian.org/debian/ stretch main
deb-src http://deb.debian.org/debian/ stretch main
deb http://security.debian.org/ stretch/updates main
deb-src http://security.debian.org/ stretch/updates main
deb http://deb.debian.org/debian/ stretch-updates main
deb-src http://deb.debian.org/debian/ stretch-updates main
deb http://ftp.debian.org/debian stretch-backports main
</code></pre></div> </div>
</li>
<li><code class="language-plaintext highlighter-rouge">sudo apt update</code></li>
<li><code class="language-plaintext highlighter-rouge">sudo apt install python-certbot-apache -t stretch-backports</code></li>
<li><code class="language-plaintext highlighter-rouge">sudo certbot --apache -d jenkins.abdelrahmanhosny.me</code> and replace the domain with your domain name.</li>
</ol>
<p><strong>CHECK:</strong> If you go to your web browser page for your domain and refresh it, you should see it being served under HTTPS now.</p>
<h2 id="configure-jenkins">Configure Jenkins</h2>
<ol>
<li>Obtain the initial password: <code class="language-plaintext highlighter-rouge">sudo cat /var/lib/jenkins/secrets/initialAdminPassword</code> in the terminal.</li>
<li>Use the password to start the configuration wizard in the browser. Use the default settings for the setup, unless you know what you are looking for.</li>
</ol>
<p>Congratulations! You have a production-ready Jenkins master node.</p>
<h3 id="install-blue-ocean">Install Blue Ocean</h3>
<p>Jenkins is mainly a very powerful automation server. However, when it was started, it wasn’t very CI friendly. However, thanks to its plugin ecosystem, great plugins made common tasks easy. The most important one is <em>Blue Ocean</em> plugin. It makes defining a CI pipeline from a git repository a simple task.</p>
<ol>
<li>In Jenkins dashboard, choose <code class="language-plaintext highlighter-rouge">Manage Jenkins</code> from the left sidebar.</li>
<li>Click <code class="language-plaintext highlighter-rouge">Manage Plugins</code>.</li>
<li>Select <code class="language-plaintext highlighter-rouge">Available</code> tab.</li>
<li>Search for <code class="language-plaintext highlighter-rouge">blue ocean</code> from the search box in the top right.</li>
<li>Select <code class="language-plaintext highlighter-rouge">Blue Ocean</code> and click <code class="language-plaintext highlighter-rouge">Download now and install after restart</code>.</li>
</ol>
<h3 id="link-to-github">Link to GitHub</h3>
<ol>
<li>In the dashboard, click <code class="language-plaintext highlighter-rouge">Open Blue Ocean</code> from the left sidebar.</li>
<li>Choose GitHub from the available integration options.</li>
<li>Since you don’t have an access key, click the link below the box to obtain an access key from GitHub with the permissions that Jenkins need.</li>
<li>Save this key locally in a safe place. We will use it again shortly.</li>
</ol>
<h2 id="build-a-pipeline">Build a Pipeline</h2>
<p>Select your repository and create a pipeline. If your repository doesn’t have a Jenkinsfile already, the wizard will offer you to create one using its nice step-by-step guide.</p>
<h3 id="trigger-on-git-pushes">Trigger on Git Pushes</h3>
<p>If you push code to your repository, the pipeline won’t start automatically. The reason is Jenkins couldn’t create a webhook on GitHub to be notified if new changes are pushed to the repo. In order to fix this, follow the following steps</p>
<ol>
<li>In Jenkins dashboard, choose <code class="language-plaintext highlighter-rouge">Manage Jenkins</code> from the left sidebar.</li>
<li>Click <code class="language-plaintext highlighter-rouge">Configure System</code>.</li>
<li>Under GitHub Section, click <code class="language-plaintext highlighter-rouge">Add</code></li>
<li>From the <code class="language-plaintext highlighter-rouge">kind</code> menu, select <code class="language-plaintext highlighter-rouge">Secret text</code></li>
<li>Paste in the GitHub access key (that we saved previously) in the Secret field.</li>
<li>Select <code class="language-plaintext highlighter-rouge">Manage hooks</code> and then click <code class="language-plaintext highlighter-rouge">Test connection</code>.</li>
<li>If connection is successful, <code class="language-plaintext highlighter-rouge">save</code> the settings.</li>
<li>Now, newly created pipelines from the Blue Ocean interface will automatlically add a web hook to your GitHub repo. However, you might need to re-create the pipeline for the repo you added before in order to create a web hook.</li>
</ol>
<h3 id="git-submodules">Git Submodules</h3>
<p>If your repository uses Git submodules, an additional configuration step needs to be done after the pipeline of the repository is added through Blue Ocean interface.</p>
<ol>
<li>Go back to main Jenkins dashboard, and click on the project.</li>
<li>Select <code class="language-plaintext highlighter-rouge">Configure</code> from the left sidebar.</li>
<li>Under <em>Branch Sources</em>, click on <code class="language-plaintext highlighter-rouge">Add</code></li>
<li>Select <code class="language-plaintext highlighter-rouge">Advanced sub-modules behaviours</code></li>
<li>Select <code class="language-plaintext highlighter-rouge">Recursively update submodules</code> and <code class="language-plaintext highlighter-rouge">Use credentials from default remote of parent repository</code>.</li>
<li>Click <code class="language-plaintext highlighter-rouge">save</code> and re-run the pipeline.</li>
</ol>
<h2 id="summary">Summary</h2>
<p>In this short tutorial, we created a production-ready Jenkins master node on a free GCP instance. It works well for small CI pipelines. Following the same steps on a bigger VM instance makes the Jenkins server capable of handling serious tasks in modern CI pipelines.</p>Abdelrahman Hosnyabdelrahman@brown.eduContinuous Integration (CI) proved to be of extreme importance to developer productivity. Implementing a CI pipeline sets the framework for code contributors to make sure their new additions/deletions improve the code base and don’t fall back to a non-ending journey of debugging. When it comes to implementing the pipeline, there are many cloud-based tools that can make this automation easier. For example, CircleCI, Travis CI, GitLab CI and the newly-introduced (at the time of writing) GitHub Actions. However, there are some cases where a project needs to self-host its automation server, and this is where Jenkins comes as a well-developed and well-maintained automation server. In this post, I show how to correctly set up a prodcution-level master Jenkins server. This demo is implemented on Google Cloud Platform as it offers unlimited free (forever or until further notification) Compute Engine instance. It is a small instance (F-1 micro), but should be fine for small tasks. If you have builds that take longer times, or requires a large amount of memory, you should start a bigger server on any cloud provider. Also, we will use Debian GNU/Linux 9 as the OS on whichever instance you choose.Caching Builds with Docker Build Branching2019-09-19T00:00:00+00:002019-09-19T00:00:00+00:00https://abdelrahmanhosny.me/tech/this-does-not-work/failure-story-docker-branching<p><strong>GitHub</strong>: https://github.com/abdelrahmanhosny/yosys</p>
<p><strong>Description</strong>: <em>Yosys</em> is an open-source Logic Synthesis tool. It takes a circuit design written in Register-Transfer-Level (RTL) and synthesize it to produce a netlist ready for placement and routing. The project is written in C++. The tool is used in an end-to-end silicon compilation flow, called <a href="https://github.com/The-OpenROAD-Project">OpenROAD</a>. The complete flow unifies a build platform to use <code class="language-plaintext highlighter-rouge">CentOS</code>. Therefore, it was ideal to introduce a <code class="language-plaintext highlighter-rouge">Docker</code> build for it to ensure portability.</p>
<!--more-->
<p>The <a href="https://github.com/abdelrahmanhosny/yosys/blob/master/Dockerfile">Dockerfile</a> uses a multi-stage build to produce a final image of minimal size.</p>
<pre><code class="language-Docker">FROM centos:centos6 AS builder
# install gcc 7
RUN yum -y install centos-release-scl && \
yum -y install devtoolset-7 devtoolset-7-libatomic-devel
ENV CC=/opt/rh/devtoolset-7/root/usr/bin/gcc \
CPP=/opt/rh/devtoolset-7/root/usr/bin/cpp \
CXX=/opt/rh/devtoolset-7/root/usr/bin/g++ \
PATH=/opt/rh/devtoolset-7/root/usr/bin:$PATH \
LD_LIBRARY_PATH=/opt/rh/devtoolset-7/root/usr/lib64:/opt/rh/devtoolset-7/root/usr/lib:/opt/rh/devtoolset-7/root/usr/lib64/dyninst:/opt/rh/devtoolset-7/root/usr/lib/dyninst:/opt/rh/devtoolset-7/root/usr/lib64:/opt/rh/devtoolset-7/root/usr/lib:$LD_LIBRARY_PATH
# python 3.6
RUN yum -y install rh-python36
ENV PATH=/opt/rh/rh-python36/root/usr/bin:$PATH
# install other yosys dependencies
RUN yum install -y flex tcl tcl-devel libffi-devel git graphviz readline-devel glibc-static wget autoconf && \
wget https://ftp.gnu.org/gnu/bison/bison-3.0.1.tar.gz && \
tar -xvzf bison-3.0.1.tar.gz && \
cd bison-3.0.1 && \
./configure && \
make -j$(nproc) && \
make install
COPY . /yosys
WORKDIR /yosys
RUN make PREFIX=build config-gcc-static-tcl-dynamic
RUN make PREFIX=build -j$(nproc)
RUN make PREFIX=build install
FROM centos:centos6 AS runner
RUN yum update -y && yum install -y readline-devel tcl-devel libffi-devel
COPY --from=builder /yosys/build/bin/yosys /build/yosys
COPY --from=builder /yosys/build/bin/yosys-abc /build/yosys-abc
COPY --from=builder /yosys/build/bin/yosys-config /build/yosys-config
COPY --from=builder /yosys/build/bin/yosys-filterlib /build/yosys-filterlib
COPY --from=builder /yosys/build/bin/yosys-smtbmc /build/yosys-smtbmc
RUN useradd -ms /bin/bash openroad
USER openroad
WORKDIR /home/openroad
</code></pre>
<h2 id="problem">Problem</h2>
<p>The problem with this Dockerfile is it cannot be used for a development environment, where a contributor: modify code -> build -> test -> repeat. The reason is in this line <code class="language-plaintext highlighter-rouge">COPY . /yosys</code>. Everytime there is a new change in of the code files, a whole new copy to the Docker image is executed when issuing <code class="language-plaintext highlighter-rouge">docker build -t <image_name> .</code> This is time-wasting for the developer.</p>
<p>Why not use a local development environment? That’s a possible solution. In our specific use case, the development environment changes frequently from a contributor to another. We wanted to implement a development environment based on Docker. Especially, the Docker-based build will be utilized in a Continuous Integration (CI) pipeline.</p>
<p><strong>So, how can we cache the build directory?</strong></p>
<h2 id="attempted-solution">Attempted Solution</h2>
<p>I wanted to modify the Dockerfile to implement this logic:</p>
<pre><code class="language-Docker">FROM centos:centos6
RUN …
IF BUILD_TARGET==development
// cache build directory
ELSE IF BUILD_TARGET==production
// use multi-stage build to get a small final image size.
DONE
</code></pre>
<p>However, <code class="language-plaintext highlighter-rouge">Dockerfiles</code> don’t offer conditional branching! How can we achieve this functionality?</p>
<p><a href="https://docs.docker.com/engine/reference/builder/#arg">Docker Build Arguments</a> defines a variable that we can pass at build-time to the builder with the <code class="language-plaintext highlighter-rouge">docker build</code> command using the <code class="language-plaintext highlighter-rouge">--build-arg <varname>=<value></code> flag. Utilizing this feature, one can implement a branching model for Dockerfiles. The following model is inspired from <a href="https://medium.com/@tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae">this post</a>.</p>
<h3 id="dockerfile-branching-model">Dockerfile Branching Model</h3>
<p>The attempt is to implement the following scenario. <a href="/assets/img/posts/failure-story-docker-branching/Dockerfile-branching.svg" target="_blank">Click here</a> to open the image in a new tab and Zoom in.</p>
<p><img src="/assets/img/posts/failure-story-docker-branching/Dockerfile-branching.svg" alt="" /></p>
<p>Here, we utilize two build arguments that we are going to branch upon:</p>
<ol>
<li><code class="language-plaintext highlighter-rouge">BUILD</code>: [cached, notcached]</li>
<li><code class="language-plaintext highlighter-rouge">ENVIRONMENT</code>: [production, development]</li>
</ol>
<p><strong>The build flow goes as follows:</strong></p>
<ul>
<li>In the first phase (yellow boxes), we build a stage that is not cached. We build another stage that is based on the cached stage and it basically copies the build directory from the <code class="language-plaintext highlighter-rouge"><image></code> built previously (using <code class="language-plaintext highlighter-rouge">noncached</code> argument).</li>
<li>Based on the <code class="language-plaintext highlighter-rouge">BUILD</code> argument, we introduce a new build stage that is based on one of the previous stages (cached or notcached). This branching is represented in the orange box. And this is also how we implement branching in Dockerfiles.</li>
<li>Now, from the builder image, we build two new stages: development and production. The development stage copies everything from the builder stage (source code + build directory). The production stage copies only the binaries needed to run the tool at the end. These stages are represented in the red box.</li>
<li>Based on the <code class="language-plaintext highlighter-rouge">ENVIRONMENT</code> argument, we introduce a new build stage that is based on one of the two previous stages (development or production). This branching is represented in the final green box.</li>
</ul>
<p>The idea here is to first build using a <code class="language-plaintext highlighter-rouge">noncached</code> and <code class="language-plaintext highlighter-rouge">development</code> environment. This yields a final Docker image that contains the build directory in it, in addition to the source code directory. The command used for this is: <code class="language-plaintext highlighter-rouge">docker build --build-arg BUILD=noncached --build-arg ENVIRONMENT=development yosys .</code>. Next, a developer will build using <code class="language-plaintext highlighter-rouge">cached</code> value for the <code class="language-plaintext highlighter-rouge">BUILD</code> argument (<code class="language-plaintext highlighter-rouge">docker build --build-arg BUILD=cached --build-arg ENVIRONMENT=development yosys .</code>). This means that the build is based on the previous build directory that is already there in the image. In our CI pipeline, and when we want to release the tool, we would use <code class="language-plaintext highlighter-rouge">ENVIRONMENT=production</code>.</p>
<p>Note that the Docker Build Engine does not build all stages. It determines which stages are needed for the final image and recursively traverses the stage dependencies in the Dockerfile. This is neat!</p>
<h3 id="resulting-dockerfile">Resulting Dockerfile</h3>
<p>The final Dockerfile using this branching model is shown below:</p>
<pre><code class="language-Docker">ARG BUILD=notcached
ARG ENVIRONMENT=production
FROM centos:centos6 AS builder-notcached
# install gcc 7
RUN yum -y install centos-release-scl && \
yum -y install devtoolset-7 devtoolset-7-libatomic-devel
ENV CC=/opt/rh/devtoolset-7/root/usr/bin/gcc \
CPP=/opt/rh/devtoolset-7/root/usr/bin/cpp \
CXX=/opt/rh/devtoolset-7/root/usr/bin/g++ \
PATH=/opt/rh/devtoolset-7/root/usr/bin:$PATH \
LD_LIBRARY_PATH=/opt/rh/devtoolset-7/root/usr/lib64:/opt/rh/devtoolset-7/root/usr/lib:/opt/rh/devtoolset-7/root/usr/lib64/dyninst:/opt/rh/devtoolset-7/root/usr/lib/dyninst:/opt/rh/devtoolset-7/root/usr/lib64:/opt/rh/devtoolset-7/root/usr/lib:$LD_LIBRARY_PATH
# python 3.6
RUN yum -y install rh-python36
ENV PATH=/opt/rh/rh-python36/root/usr/bin:$PATH
# install other yosys dependencies
RUN yum install -y flex tcl tcl-devel libffi-devel git graphviz readline-devel glibc-static wget autoconf && \
wget https://ftp.gnu.org/gnu/bison/bison-3.0.1.tar.gz && \
tar -xvzf bison-3.0.1.tar.gz && \
cd bison-3.0.1 && \
./configure && \
make -j$(nproc) && \
make install
COPY . /yosys
FROM builder-notcached AS builder-cached
COPY --from=openroad/yosys /yosys/build /yosys/build
FROM builder-${BUILD} AS builder-final
WORKDIR /yosys
RUN make PREFIX=build config-gcc-static-tcl-dynamic
RUN make PREFIX=build -j$(nproc)
RUN make PREFIX=build install
FROM centos:centos6 AS runner-development
COPY --from=builder-final /yosys /yosys
FROM centos:centos6 AS runner-production
RUN yum update -y && yum install -y readline-devel tcl-devel libffi-devel
COPY --from=builder-final /yosys/build/bin/yosys /build/yosys
COPY --from=builder-final /yosys/build/bin/yosys-abc /build/yosys-abc
COPY --from=builder-final /yosys/build/bin/yosys-config /build/yosys-config
COPY --from=builder-final /yosys/build/bin/yosys-filterlib /build/yosys-filterlib
COPY --from=builder-final /yosys/build/bin/yosys-smtbmc /build/yosys-smtbmc
FROM runner-${ENVIRONMENT}
RUN useradd -ms /bin/bash openroad
USER openroad
</code></pre>
<h2 id="what-went-wrong">What Went Wrong?</h2>
<p>The same source of problem: <code class="language-plaintext highlighter-rouge">COPY . /yosys</code>. It turns out that the <code class="language-plaintext highlighter-rouge">COPY</code> command of Dockerfile does <strong>NOT</strong> preserve all metadata of files (especially timestamps). This means that when we <code class="language-plaintext highlighter-rouge">COPY</code> files to the Docker image being created, all files get a fresh new <code class="language-plaintext highlighter-rouge">last modified</code> date. This basically breaks all assumptions I mistakenly made in the beginning. So, the <code class="language-plaintext highlighter-rouge">COPY</code> command not only copies everything in the source directory if <strong>one</strong> file changed, but also gives a fresh new timestamp to all files! meh!</p>
<p>Although there is an option in the <code class="language-plaintext highlighter-rouge">COPY</code> command that changes the ownership of the copied content (<a href="https://docs.docker.com/engine/reference/builder/#copy">see reference</a>), there is no flag to preserve timestamps. There was a proposal to come up with a solution around this problem. See <a href="https://github.com/moby/moby/issues/13982">Ability to filter ADD / COPY during docker build, based on DIFF</a>. But the issue got closed for reasons in the discussion thread there.</p>
<h2 id="did-i-ever-cache-results-with-docker">Did I Ever Cache Results with Docker?</h2>
<p>Yes! I implemented two Dockerfiles. One called <code class="language-plaintext highlighter-rouge">Dockerfile.dev</code> that only builds the dependencies of the tool. Then, in order to build the actual tool, I do it during <code class="language-plaintext highlighter-rouge">docker run</code> and mount the source directory inside the container. The <code class="language-plaintext highlighter-rouge">build</code> directory is preserved on the host after the <code class="language-plaintext highlighter-rouge">docker run</code> finishes. The other file is the <code class="language-plaintext highlighter-rouge">Dockerfile</code> presented in the beginning of this post, which is used for release purposes. The final Docker command for build looks something like: <code class="language-plaintext highlighter-rouge">docker run -v $(pwd):/yosys yosys bash -c 'cd /yosys && make'</code></p>
<h2 id="now-what">Now What?</h2>
<p>Don’t try to over-engineer a solution like the one presented above. The force of nature is stronger than any developer! This post is an attempt to write about failures that waste developers time, similar to how we write about our successes. At the end, they both try to save some other developer’s time.</p>
<p>If you feel this will save someone else’s time, share it with them :)</p>Abdelrahman Hosnyabdelrahman@brown.eduGitHub: https://github.com/abdelrahmanhosny/yosys Description: Yosys is an open-source Logic Synthesis tool. It takes a circuit design written in Register-Transfer-Level (RTL) and synthesize it to produce a netlist ready for placement and routing. The project is written in C++. The tool is used in an end-to-end silicon compilation flow, called OpenROAD. The complete flow unifies a build platform to use CentOS. Therefore, it was ideal to introduce a Docker build for it to ensure portability.International Students - Understand USA Tax Jargon2019-08-31T00:00:00+00:002019-08-31T00:00:00+00:00https://abdelrahmanhosny.me/random/international-students-usa-taxes<p>Today, I was asked by a friend who is a new comer to a graduate-level program: <em>Can you give me any guidance about taxes? Do we have any sort of treaties for reduction, or how did you manage taxes here? What should I expect to pay in general?</em></p>
<p>Rephrasing his question, <em>What is the deal with all this tax bullshit they told us about in the student orientation?</em></p>
<p>So, I replied to his email and my reply looked so long that I thought it would benefit some confused new comer out there. While this is not a financial advice, I just describe what the terms you will hear/read everywhere mean.</p>
<h2 id="tldr">TL;DR</h2>
<p>Don’t get scared and don’t worry about taxes. You will know when it is time to take action.</p>
<h2 id="some-types-of-taxes">Some Types of Taxes</h2>
<p>If you are coming from a country that didn’t require you to sort out your taxes (just like my country - Egypt), you will most probably be confused similar to how I was in my first few days in the US. You go to a grocery store to buy something, and at the cashier, you find yourself paying a little bit more than the price tag on the item. Welcome to <strong>Sales Taxes</strong>.</p>
<!--more-->
<p>If you are coming from a country that you can live in without a car and feel free to navigate a city using public transportation or a bike, you will most probably be challenged in the US, especially if your school is in a city that doesn’t have basic transportation services. If you drive, your first thought would be: <em>I will buy a car</em>. Welcome to <strong>Property Taxes</strong>.</p>
<p>If you are sponsored by your university (e.g. Fellowship, Research Assistantship, Teaching Assistantship) or a US-based scholarship (e.g. Fulbright), you basically have US income. And for that, I would like to welcome you to <strong>Income Taxes</strong> - the largest headache.</p>
<p class="message"><strong>NOTE</strong>: There are other types of taxes that you might not need to know now.</p>
<h2 id="sales-taxes">Sales Taxes</h2>
<ul>
<li>These are taxes collected by the state you live in on <strong>any purchase transaction</strong> that happen within its borders.</li>
<li>Everything is taxed. Food, clothes, electronics, services, …etc.</li>
<li>It can be 0% in some states, but ranges between ~2% to ~10% in most of the other states. <a href="https://www.usatoday.com/story/money/taxes/2018/03/27/states-highest-and-lowest-sales-taxes/452512002/">See this map</a>.</li>
<li>Only <strong>unprocessed</strong> food and food surplus are not taxed. For example, raw meat, vegetables, essential oils, and so on. They are considered a life necessity and are note taxed.</li>
<li>If you are buying from local groceries, you will find that whatever the price you see is what you pay at the end. Walmart, Stop and Shop, Safeway, QFS and all other big grocery shops will tax almost everything (even raw food) as they are considered processed/packaged food.</li>
<li>But what if I’m buying from online service like Amazon? How is tax calculated? Answer: it is complicated. <a href="https://www.amazon.com/gp/help/customer/display.html?nodeId=202036190">See this</a></li>
</ul>
<p class="message"><strong>ACTION</strong> You don’t have to keep track of these taxes. You will pay them anyway.</p>
<h2 id="property-taxes">Property Taxes</h2>
<ul>
<li>These are taxes collected by the state on selected possessions that you own within the state borders.</li>
<li>They are collected yearly, and you will get a mail at home.</li>
<li>Basically, when you buy a car, you will pay yearly taxes - <strong>PLUS</strong> the sales tax at the time of the purchase. If you are buying from a private party (not from a dealer), you will pay the sales tax at the DMV when you register the vehicle.</li>
<li>If you own a house, a store, a garage, a backyard, …etc, you will pay as well. But I don’t think as an international student, you have any of these.</li>
</ul>
<p class="message"><strong>ACTION</strong> You don’t have to worry about these taxes. You will know when you have to pay.</p>
<h2 id="income-taxes">Income Taxes</h2>
<p>The source of all headache. Let’s go!</p>
<p>There are two entities that come after you <strong>yearly</strong> and take a piece of your cake (income). Let me introduce you to:</p>
<ul>
<li><strong>The Federal Government</strong>: represented by the Internal Revenue Service (IRS). Remember this name as you will find it everywhere.</li>
<li><strong>The State Government</strong>: represented by the Department of Revenue Service (DRS) or Deparatment of Taxation (TX), or Department of Revenue, or … oh!
Each state has its own name for their representative.</li>
</ul>
<h3 id="what-cake-are-they-cutting-from">What cake are they cutting from?</h3>
<p>Your personal income.</p>
<p><strong>A Personal Income</strong> is any money that you receive during the year. Sources include a full-time job, a part-time job, a consultancy job, awards, fellowships, scholarships, bank account interest, investments earnings, driving for Uber, selling services online, or basically anything that gets in your pocket (or bank account) and qualifies as personal income.</p>
<p class="message"><strong>NOTE</strong>: A friend sending you money to pay for something is not a personal income.</p>
<p><strong>A Year</strong> is calculated from January 1st to December 31st.</p>
<p class="message"><strong>CAUTION:</strong> If you have American Citizenship or permanent residency (green card), foreign income <strong>must</strong> also be included. That includes income to any bank account outside the US.</p>
<h3 id="so">So?</h3>
<ul>
<li>If you are not a resident, your visa (which is expected to be F-1 or J-1) limits your income options. From all the above sources, <strong>you can only work for the sponsoring institution (your university)</strong> and you <strong>CANNOT</strong> receive money from any other job, unless it is permitted on your I-20 or DS-2019 (e.g. internships).</li>
</ul>
<p class="message"><strong>TIP</strong>: when in doubt about some money being transferred to you, seek financial advice BEFORE the transaction is posted.</p>
<p class="message"><strong>TIP</strong>: Don’t give your social security numbner (SSN) to any other entity except the sponsoring institution.</p>
<ul>
<li>Your employer (sponsoring institution) maintains a record for all the money that it transfers to you during the year.</li>
<li>In January <strong>the year after</strong>, your employer send you a form called <strong>W-2</strong>.</li>
<li>The W-2 form includes every penny that got transferred to you, and it marks if there are any taxes deducted for the federal <strong>and/or</strong> state government on behalf of you.</li>
<li><strong>Tax reporting season</strong> starts from January to Mid April for people to report their income for <strong>the preceding year</strong>.</li>
<li>You collect all your W-2 forms from your employer(s) and start seeing how much you got paid and how much of that was already held for federal or state taxes.</li>
<li>This is done by filling some forms, which tell you at the end if you still owe money to the government to pay or your employer actually overpaid your taxes and you should get some tax refund.</li>
<li>You usually prepare these forms through a certified accountant (e.g. companies like H&R Block), or use online software tools.</li>
<li>Some universities provide a free tool to prepare your federal tax forms <strong>but not</strong> the state tax forms. For state taxes, another paid service is called Sprintax can be used.</li>
<li>Depending on your source of fund, some institutions don’t hold any taxes on behalf of you. Some are required to do so.</li>
</ul>
<p class="message"><strong>TIP</strong>: Once you check in to your school, ask if taxes will be deducted or not. If not, you probably want to save some amount to pay during the tax season. You will know how to calculate it shortly.</p>
<ul>
<li>In any case, you have to report your income during the tax season and see how much you owe the government or vice versa.</li>
</ul>
<h3 id="how-to-prepare-the-tax-forms">How to prepare the tax forms?</h3>
<p>Do yourself a favor and find a certified service to fill in the forms for you <strong>and co-sign</strong> them, especially in the first year. Tax forms are complicated, and they are complicated on purpose. They don’t want people to mess up with them and report false income. You also don’t want to get audited years later. It might be bad for your immigration record.</p>
<h2 id="tax-benefits">Tax Benefits</h2>
<ul>
<li>There are tax treaties between the US and some other countries. Basically, a tax treaty says: the federal government will <strong>NOT</strong> collect taxes on the first $XXX.XX income in a given year. This holds true for <strong>the first</strong> <code class="language-plaintext highlighter-rouge">N</code> years you stay in the US. After <code class="language-plaintext highlighter-rouge">N</code> years, you can’t get this treaty. Example, if the tax treaty is for $3,000 and your yearly income has been $20,000, it means that the federal government will take taxes on $17,000. It does not mean that you are getting $3,000 in taxes back.</li>
<li>Your sponsoring institution might consider this for the first payment if you marked that you qualify for a treaty when you check in with their business office. Otherwise, they will tax this income and you will figure it out when you do your tax reporting.</li>
<li>If you have a family (spouse and/or children), you include them in your tax reporting during the tax season as you are considered responsible for the family and you shouldn’t pay the same taxes as a single person. You will most probably find that the government owe you money and will get a refund.</li>
<li>Any other form of tax benefit you see or read online (e.g. school supplies, medicare, ..etc.) <strong>does NOT apply to foreign citizens</strong>; only applies to Americans. So, don’t spend time looking for what can get you more money back. And don’t get scammed by accountants during the tax season who say that they can do the taxes for you and get you more refund than the other company. They don’t know what forms to fill in.</li>
</ul>
<h2 id="but-how-much-do-governments-tax-us">But how much do governments tax us?</h2>
<ul>
<li>Depends on the personal income and the family that you are responsible for.</li>
<li>See <a href="https://smartasset.com/taxes/income-taxes">https://smartasset.com/taxes/income-taxes</a> to get an idea.</li>
</ul>
<h2 id="summary">Summary</h2>
<p>In January of every year, you will get a form that says how much you earned and you will get instructions on how to file your tax forms for the federal and state governments. Forget about the above text.</p>
<p>If you find this post useful, share with a friend :)</p>Abdelrahman Hosnyabdelrahman@brown.eduToday, I was asked by a friend who is a new comer to a graduate-level program: Can you give me any guidance about taxes? Do we have any sort of treaties for reduction, or how did you manage taxes here? What should I expect to pay in general? Rephrasing his question, What is the deal with all this tax bullshit they told us about in the student orientation? So, I replied to his email and my reply looked so long that I thought it would benefit some confused new comer out there. While this is not a financial advice, I just describe what the terms you will hear/read everywhere mean. TL;DR Don’t get scared and don’t worry about taxes. You will know when it is time to take action. Some Types of Taxes If you are coming from a country that didn’t require you to sort out your taxes (just like my country - Egypt), you will most probably be confused similar to how I was in my first few days in the US. You go to a grocery store to buy something, and at the cashier, you find yourself paying a little bit more than the price tag on the item. Welcome to Sales Taxes.