mrbusche.com

Architecture Modernization

• Rather than treating teams as feature factories (feeding them predefined solutions), giving them outcomes to achieve and the freedom to discover solutions in their subdomains unlocks more of their creative talents.

• As renowned product management expert and author Marty Cagan says, “If you’re just using your engineers to code, you’re only getting about half their value…. The best single source for innovation is your engineers”

• Internal developer platforms (IDPs) are another crucial component to achieving IVSS. IDPs reduce the friction of building, deploying, and supporting code by providing an exceptional developer experience (DX) through the use of concepts like paved roads/golden paths. This allows stream-aligned teams to focus on business outcomes without being bogged down by extraneous infrastructure-related activities and development friction.

• Modernization isn’t just rewriting the old system with new technologies. It’s an opportunity to completely rethink the UX, product functionality, business processes, and domain model and remove unneeded complexity.

• The importance of ongoing learning and upskilling cannot be overstated. It is perhaps the most important part of architecture modernization. If teams don’t have the time and opportunity to learn and practice modern concepts, there is a serious risk that new architecture will be designed with the old ways of thinking, and many of its flaws will be carried across.

• Architecture modernization is about converting dated architecture, which is a business liability, into modern architecture that provides a competitive advantage.

• Modernization requires short-term compromises for long-term prosperity, which is why leaders are reluctant to commit, but this creates a negative spiral where the architecture becomes even more of a liability.

• As more of the world is run by software, systems will become more complex, and architecture will become even more important.

• Modernization does not happen without learning new skills and acting differently. Leaders need to be aware that a significant investment is necessary for supporting every employee involved in modernization to ensure they have the required skills. If not, modernization may take far longer, or the new architecture may look just like the old one or even worse. Learning and upskilling (the topic of chapter 17) is not a onetime workshop or training course; it’s an ongoing financial and time investment. Learning needs to be built into the organization’s culture.

read more

Open Talent by John Winsor

• The post-Covid era has been an even bigger black swan, one that calls into question all our assumptions about how the world is supposed to work. According to Taleb, black swan events contain three elements:

  • The element of surprise, which catches everyone off guard
  • Impacts and outcomes that are substantial, with potentially global repercussions
  • The appearance of inevitability after the fact, given all the relevant signals and data
• Leaders who want to change their own minds may want to consider an important reframing of recent events. The millions of dissatisfied employees who are joining the Great Resignation aren’t rejecting work.

• They are rejecting jobs that pay them less than they feel they are worth and that constrain their creativity and stifle their potential. They are looking for ways to do more while doing better for themselves. The organizations that have been experimenting with open talent strategies and outside in innovation are capturing them and, as a result, continuing to push forward and succeed.

• How to change your mind.

  • Disruption offers new opportunities for grand change, especially since the supply of talent has already made the shift. Companies must begin to adapt to talent because talent is no longer adapting to corporations. Platforms fundamentally change how workers engage with their employers and vice versa, shifting the power structure so that talent has the edge. By removing the idea of a physical workspace, open talent is no longer hampered by office politics and can thrive, focused on the things that motivate people the most.
• Companies that can move quickly enough to profit from change don’t bog themselves down with a ton of bureaucracy when they set out to innovate. They don’t micromanage the process from the top down, and they don’t insist on owning all the means they employ to succeed. Their teams are light on their feet, have a sense of ownership over outcomes, and are empowered to make their own decisions and recruit all the help they need.

What Are the Goals of Your Center of Excellence?

• A COE’s overarching goal should be to take full advantage of all the talent opportunities that are available. Every organization is unique, but we suggest you start with these three principles:

  • Understand your readiness. While many people inside your organization may be using open talent already, there can be, as we’ve seen, many blind spots and roadblocks when you try to institutionalize it. The better you understand your readiness, the better your chances of a successful adoption.
  • Be prepared; things evolve rapidly. Understand that the ecosystem of open talent is quickly evolving, just as technology is. With the emergence of VDI (virtual desktop infrastructure), there has been a profusion of solutions to compliance concerns and security issues. These solutions dramatically remove the friction of adoption. A COE is necessary to stay on top of these changes and be ready to implement those that best address the needs of the organization.
  • Fit your digital transformation to your talent strategy. In many companies, talent acquisition is siloed between HR, procurement, and innovation. The COE should digitally transform all these silos by using internal and external platforms.
• Remember that open talent is about removing friction. It frees your people from the bureaucratic encumbrances that prevent them from moving faster than your competitors, from getting help from outsiders when they need it, and from capturing and applying the wisdom of crowds to the toughest problems. Companies that use those capabilities are like boxers, bobbing and weaving through punches, agilely sidestepping disruptions, and continually floating and testing new ideas and approaches.

• Accordingly, when developing your open talent strategy, keep in mind the following elements: responsiveness, agility, speed, efficiency, commitment, and staying emergent. These are attributes as opposed to goals; think of them as benchmarks to guide your decisionmaking.

• Assess. Before you can figure out where you’re going, you must know where you are. By asking the right questions, the COE leads the assessment process by helping leaders understand the organizational changes that are needed to develop open talent solutions.

• Learn. This phase aims to use education, culture, and communications to create a coalition of the curious and willing. It’s easy, especially for talent innovators, to race ahead quickly with new ideas or models. But such speed only works when a single entrepreneur is leading the charge or needs convincing. When the proposed changes are companywide, people need to understand what they are getting into and why. The COE lays the groundwork for the program through workshops and formal and informal communications.

• When companies encourage and support mobility within the organization, employees feel more valued and empowered to develop their skills continually-and therefore more likely to stay on the job.

• A company can lose good open talent candidates if it has rigid onboarding processes, especially those that take twelve weeks or longer. If at all possible, rearchitect your internal procedures such as background checks, drug testing, and IT security protocols to cut down on time–or allow the platform to carry out some of this due diligence.

• The biggest opportunity for you when you start your open talent journey is to tap into your current team to help you solve problems and do the tasks you need. There is always a combination of cognitive surplus and people inside your organization looking for mobility with upskilling and project work.

• Our research shows that, using platforms, it typically takes four days to hire a freelancer to do the work you need instead of the average two plus months to find the right talent in this talent-constrained environment through traditional methods. We also find that the freelance talent you hire on a platform is typically 30 percent less expensive and 40 percent more productive than the internal employee performing the same tasks. You see this massive gain in productivity because you’re hiring freelancers to do a task, not to play a role in the company. You are paying them to do the work you want, not to spend time at corporate meetings, office gatherings, and the like.

• External talent clouds for strategic advantages, cost savings, and flexibility. The ETC is a term we use for this growing practice; it’s an efficient and cost-effective way that companies can obtain the specialized talents they need from outside sources without sacrificing quality or productivity.

• How does an ETC work? The key to success in this phase rests on strong and trusting relationships with the right platform partners. Our recommendation is to take a two-fold approach. Revisit how you managed your full-time staff and determine which of these roles are better suited for freelance workers. The goal here should be to become agile yet to remain focused on the indispensability of certain positions for the company’s operations to run smoothly.

• Based on our analysis of research conducted by Michael Menietti and Karim Lakhani on Topcoder contest submissions across a range of software development projects, we’ve found that you don’t need hundreds of submissions to obtain that extreme value. You only need about twenty-two entrants.

• developing innovation strategies, firms must consider whether to make or buy them. In general, companies do not rely on any one strategy exclusively. One study found that 72 percent of firms that describe themselves as innovative actually rely on both making strategies and buying them. But there’s a third possible approach. Innovation contests add a fresh perspective to the make-or-buy discussion.

• Since 2018, when the platform Unilever uses-flex-work-was launched, the company has unlocked half a million hours of employee engagement and has seen a 41 percent improvement in overall productivity. Flex-work offers other benefits as well, such as helping employees whose jobs are in jeopardy because of automation to seek out opportunities for reskilling.

• flex-work platform was developed by Gloat and is used by over a hundred companies globally. Its founder, Ben Reuveni, got the idea for the platform when he was working at IBM and realized that it would be easier for him to find a job at a different company than it was to take his career in a new direction at IBM

• So, they did something much simpler. Sharma started an Excel spreadsheet with projects that his team needed help with and distributed it in a weekly email throughout the technology organization. Lo and behold, those projects were suddenly getting done-one hundred of them over two years, with a success rate of 95 percent. All that it required was getting the word out within SEI, says Sharma: “Now, managers start by looking for talent through a simple internal talent mechanism before they talk about going externally. And our talent is really engaged, as they have learned to use these projects as a way to learn and expand their career opportunities.”

• Break down silos. Traditional organizations have rigid talent hierarchies in which employees’ skills are hoarded. This structure prevents the overall workforce from achieving its full potential. ITMs (Internal talent marketplace) break down silos and allow for crossfertilization.

• Improve engagement and retention. If people in your workforce don’t see a future with your organization, they’re going to find somewhere else to build one. ITMs empower employees to move laterally and vertically and to learn new skills and seek out new opportunities that align with their passion, skills, and ambition. Empowering your employees in this way helps your organization meet its goals.

• Encourage experiential learning. Traditional development efforts fall short because they fail to include experiential learning (such as offering ways to learn by doing, rather than just by reading or listening to someone else). These efforts also ignore the employee’s acquired knowledge and ability to apply new skills in a real-time setting. ITMs create a unified workforce system that can act as a single source for skills management

• ITMs are not merely the next evolution of HR technology. They transform organizations by offering employees who have a cognitive sur plus the opportunity to work on projects they desire. In effect, these marketplaces also help employees with upskilling and reskilling while allowing them to maintain a sense of innovation and opportunities for professional growth. The ITM is best used to retain employees and can be quite powerful when combined with remote work and skill-building opportunities. For the enterprise, unlocking agility, breaking down silos and capturing the cognitive surplus are just a few benefits.

read more

GitHub Actions in Action

Generating an SBOM using the Microsoft SBOM tool

name: Generate SBOM
  run: |
    curl -Lo $RUNNER_TEMP/sbom-tool https://github.com/microsoft/sbomtool/releases/latest/download/sbom-tool-linux-x64
    chmod +x $RUNNER_TEMP/sbom-tool
    $RUNNER_TEMP/sbom-tool generate -b ./buildOutput -bc . -pn Test -pv 1.0.0 -ps mycompany -nsb https://sbom.mycompany.com -V Verbose

Job summaries

Here is an example that adds Markdown and plain HTML to the job summary:
  - run: echo '### Hello world! :rocket:' >> $GITHUB_STEP_SUMMARY
  - run: echo '### Love this feature! :medal_sports:' >> $GITHUB_STEP_SUMMARY
  - run: echo '<h1>Great feature!</h1>' >> $GITHUB_STEP_SUMMARY

Built-in functions in GitHub for expressions

toJSON()
fromJSON()
hashFiles()
contains(search, item)
startsWith()
endsWith()
format() // replaces values in string

Functions to check status of workflow job

success()
always()
cancelled()
failure()

Chaining workflow jobs

job_1:

job_2:
  needs: job_1

read more

Generative Deep Learning by David Foster

• A generative model must also be probabilistic rather than deterministic, because we want to be able to sample many different variations of the output, rather than get the same output every time.

• A generative model must include a random component that influences the individual samples generated by the model.

Representation Learning

• Suppose you wanted to describe your appearance to someone who was looking for you in a crowd of people and didn’t know what you looked like. You wouldn’t start by stating the color of pixel 1 of a photo of you, then pixel 2, then pixel 3, etc. Instead, you would make the reasonable assumption that the other person has a general idea of what an average human looks like, then amend this baseline with features that describe groups of pixels, such as I have very blond hair or I wear glasses. With no more than 10 or so of these statements, the person would be able to map the description back into pixels to generate an image of you in their head.

If you use word tokens:

• All text can be converted to lowercase, to ensure capitalized words at the start of sentences are tokenized the same way as the same words appearing in the middle of a sentence. In some cases, however, this may not be desirable; for example, some proper nouns, such as names or places, may benefit from remaining capitalized so that they are tokenized independently

read more

Generative AI in Action

Not a lot of notes because a lot was things I’ve already learned. Would be a great resource for anyone new to the space.

In the past, we would need to use a named entity recognition (NER) model for entity extraction; furthermore, that model would need to have seen the data and be trained as part of its dataset. With LLM models, we can do this without any training, and they are more accurate. While traditional NER methods are effective, they often require manual effort and domain-specific customization. LLMs have significantly reduced this burden, offering a more efficient and often more accurate approach to NER across various domains. A key reason is the Transformer architecture, which we will cover in the next few chapters. This is a great example of traditional AI being more rigid and less flexible than generative AI.

Counting tokens for GPT

import tiktoken as tk

def count_tokens(string: str, encoding_name: str) -> int:
  # Get the encoding
  encoding = tk.get_encoding(encoding_name)

  # Encode the string
  encoded_string = encoding.encode(string)

  # Count the number of tokens
  num_tokens = len(encoded_string)

  return num_tokens

# Define the input string
prompt = “I have a white dog nam

# Display the number of tokens in the String
print(“Number of tokens:” , count_tokens(prompt, “cl100k_base”))
# Running this code, as expected, gives us the following output:
# python countingtokens.py
# Number of tokens: 7

read more

Observability for Large Language Models

• Organizations that lack good tools and data to understand systems in production may find themselves ill-prepared to tackle the challenges posed by a feature that uses LLMs.

• LLMs represent a step change in the capability and accessibility of machine learning (ML) models for organizations. Every product has problems to solve for its users where there is no single solution but rather a set of solutions lying on some spectrum of “correct” or “right.”

• However, the very things that make LLMs so useful also give rise to the biggest challenges. End users expect powerful capabilities with reliable behavior, but steering an LLM to reliability for all possible inputs is challenging. Furthermore, the tools that product engineers traditionally lean on for improving reliability - step-by-step debugging and unit testing aren’t feasible with LLMs.

• LLMs introduce reliability and predictability challenges that can seem scary when released to production. They are black boxes (you can’t debug them like you can a single-threaded client application) that produce nondeterministic outputs based on natural language inputs

• Natural language inputs are broad. Very broad. A natural language, such as English, is infinitely more expressive than any programming language, query language, or UI. What this means is that users of applications with natural language inputs will do things you cannot hope to predict. Yes, there will likely be patterns of similar inputs that users will input for specific reasons-you can account for those-but there is an extremely long tail of inputs your users will create, and users expect those inputs to be handled well

• Finally, attempting to de-risk a product launch through early access programs or limited user testing can introduce bias and create a false sense of security. Early access programs and user testing often fail to capture the full range of user behavior and potential edge cases that arise in real-world usage with a wide range of users.

• Put differently, you should be aware of the following things when building with LLMS:

  • Failure will happen-it’s a question of when, not if.
  • Users will do things you can’t possibly predict.
  • You will ship a bug fix that breaks something else.
  • You can’t write unit tests for LLMS (or practice test-driven development).
  • Early access programs won’t really help you.

• By combining OpenTelemetry’s automatic tracing instrumentation capabilities with manual instrumentation, you can capture everything you need to begin systematically analyzing user behavior. In this way, you can learn how user behavior impacts the results that a generative AI model will produce.

• Simple LLM Call with a Static Prompt Some prompts are mostly static. That is, they might include user input and some additional info, but inputs are largely the same each time. For these cases, you just need three spans:
  • The overall tracking span for all operations. This is important because it’s the span you’ll usually query for later on. It represents the end-to-end user experience of your feature that uses an LLM.
  • A child span of the overall tracking span that tracks the call to an LLM (this can be done with automatic instrumentation).
  • A child span of the overall tracking span that tracks any parsing or validation of outputs you do on LLM outputs.
• For many applications, these three spans will suffice for your observability needs

read more

Be Useful

• You can explain that governors don’t have control over a global financial disaster-but the truth is, you get credit when the economy’s on the way up even though you have very little to do with it, so it’s only fair that you get the blame on the way down.

• If I bring this back to my constituents, they’d say, I am going to lose my next election to someone from my own party, because they’re going to say my support of this bill is proof that I’m either not liberal or not conservative enough. I’m in a “safe seat,” they’d say, and by supporting this bill I’d be making the seat unsafe…for me.

• They were talking about the impact of being from a gerrymandered district. I was blown away when I learned how extensive gerrymandering was, not just in California but with electoral maps all over the country, at every level. And that it has been going on this way for two hundred years! When it became clear to me that one of the big reasons no meaningful legislation got done was the way electoral districts were being drawn-every ten years by the same politicians who would benefit from redrawn boundariesI knew right then that we had to fix these It became a huge goal of mine as governor.
• You would think I was trying to take away their free supply of American flag lapel pins, judging by the reaction of people from both parties when I introduced a redistricting reform measure to the ballot in 2005. No one was happy about it. A lot of politicians were pissed off.

• Everyone said it wasn’t possible, it won’t happen, I couldn’t do it.

• Let me tell you something: Nothing good has ever come from having a plan B. Nothing important or life changing, anyway. Plan B is dangerous to every big dream.
• It is a plan for failure. If plan A is the road less traveled, if it’s you carving your own path toward the vision you’ve created for your life, then plan B is the path of least resistance. And once you know that path is there, once you’ve accepted that it’s an option, it becomes so, so easy to take it whenever things get difficult. Fuck plan B! The second you create a backup plan, not only are you giving a voice to all the naysayers, but you are shrinking your own dream by acknowledging the validity of their doubts.

• Worse, you become your own naysayer. There are enough of them out there already; you don’t need to add to their ranks.

• There’s a story about Sir Edmund Hillary, the first person to summit Mount Everest. When he came back down to base camp, he was met by reporters who asked him what the view was like at the top of the world. He said it was incredible, because while he was up there he saw another mountain in the Himalayan range that he hadn’t climbed yet, and he was already thinking about the route he would take to summit that peak next.

• Fulfilling a dream gives you the power to see further and deeperfurther out into the world toward what is possible, and deeper into yourself to what you are capable of. It’s why there are so few stories about people who have done something big then just packed their bags and moved to a private island never to be heard from again. People who think big and succeed almost always continue to push and to strive and to dream bigger.

• Not surprisingly, the people who complain the most about not having enough time do the least amount of work.

• Let me put it another way: busyness is bullshit. We’re all “busy.” We all have things to do every day. Obligations and responsibilities. We all have to eat, sleep, pay the bills. What does that have to do with putting in the work to reach your vision? If it matters to you, make the time.

• When a couple or a group comes in,” he said, “you need to figure out who is in charge, who is passionate about whatever it is you’re selling, who is the one who engages the most with you. You need to know who is the customer, who is the boss, and who is making the decisions.”

• This is the missing piece of the equation. Purpose. Vi sion. We aren’t giving young people the time and space to discover a purpose or to create a vision for themselves. We aren’t allowing the world to show them what is possible for their lives. Instead, right at the point where they have the least to lose and the most to gain from spending time out in the world, we’re plucking them out of it and sticking them in four-year universities, which are the exact opposite of the real world.

read more

The Tipping Point of Change: How a Few Can Impact the Many

Forget the majority, it’s all about the minority.

In “Revenge of the Tipping Point,” Malcolm Gladwell explores the fascinating dynamics of social epidemics. He reveals how a small percentage of individuals can trigger widespread change, overturning the traditional notion that mass movements require mass participation.

The Rule of Thirds

Gladwell introduces the intriguing concept of “the magic of the third.” Through extensive research on group dynamics, he argues that a mere third of any group holds the power to drive significant shifts. This applies to everything from classroom integration to corporate boardrooms. A prime example? The transformative impact on workplace dynamics once women reached one-third representation in boardrooms.

The 1% Pollution Problem

The impact of a minority isn’t limited to social dynamics. Consider Denver’s air pollution problem. Nearly 40 years ago, Donald Stedman developed a device to measure vehicle emissions. While numerous studies highlight the potential of electric vehicles to reduce pollution, Denver uncovered a startling truth: if only the top 1% of polluting vehicles switched to electric power, the pollution reduction would be equivalent to having 10% of all cars go electric!

This isn’t to say we shouldn’t all strive to reduce our environmental footprint. However, it underscores the disproportionate impact of a small minority and the potential for rapid change if we can influence their behavior.

Super Spreaders: The COVID Case Study

The phenomenon of the influential minority extends even to the microscopic realm. A British study involving 36 volunteers, deliberately infected with COVID-19, revealed that a mere two individuals were responsible for a staggering 86% of the virus particles. This highlights the role of “super spreaders,” individuals who emit significantly more aerosol particles than average, in driving the pandemic.

The Takeaway: Focus on the Few to Impact the Many

Gladwell’s work offers a powerful lesson: change doesn’t always require convincing the masses. By identifying and influencing the key players - the “magic third,” the top polluters, the super spreaders - we can unlock a cascade of change and achieve our goals far more efficiently.

read more

Unhobbling an LLM refers to the process of removing limitations or constraints that prevent the LLM from reaching its full potential.

Imagine a really talented artist being forced to only use crayons. They could still draw some cool stuff, but imagine what they could do with paints and brushes or pencils.

The reasons for the hobbling are well intentioned. It’s kind of like keeping a powerful dog on a leash - you want to make sure things stay safe and under control.

Researchers are working on ways to “unhobble” LLMs so they can do every more amazing things. If you want to learn more about how they’re trying to make this happen check out this essay.

read more

Vector embeddings are a powerful tool for representing data in a way that machines can understand. But what if we could make them even better?

Binary quantized vector embeddings offer a way to do just that. By compressing vectors by 32x, we can achieve ~25x retrieval speedup while retaining 95+% accuracy. This means faster, more efficient search and analysis, all without sacrificing performance.

Check out this blog post to learn more about this exciting development: https://emschwartz.me/binary-vector-embeddings-are-so-cool/

read more