Matt Howard Journalist Profile

Articles

Understanding Narcissism in the Workplace: The Dual Impact of Self-Efficacy and Political Skill on Attitudes and Performance

Dec 8, 2024 | articlegateway.com | Maggie Davis |Matt Howard

DOI: https://doi.org/10.33423/jop.v24i4.7404 Keywords: organizational psychology, narcissism, self-efficacy, political skill, organizational citizenship behavior, organizational commitment, workplace deviance References Allen, N.J., & Meyer, J.P. (1990). The measurement and antecedents of affective, continuance, and normative commitment to the organization. Journal of Occupational Psychology, 63(1), 1–18. Ames, D.R., Rose, P., & Anderson, C.P. (2006). The NPI-16 as a short measure of...
Evaluating NVIDIA H200 GPUs for LLM inference

Oct 22, 2024 | baseten.co | Pankaj Gupta |Philip Kiely |Matt Howard

The NVIDIA H200 Tensor Core GPU is a high-end datacenter-grade GPU designed for AI workloads. The big sibling of the popular H100 GPU, the H200 offers more GPU memory and memory bandwidth on an equivalent compute profile. While H200 GPUs are eagerly anticipated for training, fine-tuning, and other long-running AI workloads, we wanted to see how they perform for inference tasks. With our friends at Lambda, we tested Mistral Large, a 123-billion parameter model, on an 8xH200 GPU cluster.
Poem Of The Week: Skew Hill By Alan Payne

Aug 19, 2024 | yorkshiretimes.co.uk | Alan Payne |Ian Duhig |Don Paterson |Matt Howard

artsSkew HillSheffieldMy mother’s ashesscatteredbetweenflinty showers;a resolutionat last of allthat reaching outtowards others. Her desireto be usefulshrunkto a circle of roseswhose petalsshake themselves freeof the lossthat shapesa man’s bent backin a fieldstunned by rain. A sense of emptiness inheres to the fabric of Alan Payne’s elegy for his mother.
Dreamland: Going To Ground - An Anthology Of Nature And Place, Edited By Jon Woolcott

Aug 18, 2024 | yorkshiretimes.co.uk | Ian Duhig |Don Paterson |Matt Howard

artsThe truly remarkable thing about Little Toller’s new anthology is the variegated manner of its focus. Featuring the reflections of a number of established writers, emerging from all colours of the sociocultural spectrum, the received impression is kaleidoscopic, a mingling of style with labile observation, rural with urban landscapes, as though each observer was an undirected flâneur, seduced, but not always obsessed, by themes as they pass through.
Poem Of The Week: The Courtesan By Rainer Maria Rilke (1875-1926)

Aug 9, 2024 | yorkshiretimes.co.uk | Ian Duhig |Don Paterson |Matt Howard

artsThe Courtesan The sun of Venice in my hair’s preparinga gold where lustrously shall culminateall alchemy. My brows, which emulateher bridges, you can contemplateover the silent perilousness repairingof eyes which some communion secretlyunites with her canals, so that the searises and ebbs and changes in them. Hewho once has seen me falls to envyingmy dog, because, in moments of distraction,this hand no fieriness incinerates,scathless, bejewelled, there recuperates.
Control plane vs workload plane in model serving infrastructure

May 29, 2024 | baseten.co | Colin McGrath |Matt Howard |Philip Kiely |Pankaj Gupta

Building worldwide multi-cloud AI model serving infrastructure requires powerful-but-flexible abstractions. One of the core abstractions in Basetenâ€™s infra architecture is the idea of a control plane and workload planes. Both the control and workload planes are Kubernetes clusters, but they follow a strict separation of concerns:The control plane is a single Kubernetes cluster that serves as the backend for Basetenâ€™s user interface and model management API endpoints.
Comparing tokens per second across LLMs

May 9, 2024 | baseten.co | Philip Kiely |Matt Howard |Pankaj Gupta |Phil CardyScarlet Howes

Tokens per second (TPS) is one of the most important metrics we track for LLM performance. When comparing performance across two different LLMs, you need to adjust TPS based on the modelsâ€™ tokenizers. The tokenizer is a small model that takes human-readable input text and turns it into the tokens that an LLM uses for inference. Different LLMs have different tokenizers of varying levels of efficiency:Llama 3 models use a variant of tiktoken, a tokenizer developed by OpenAI and used by GPT-4.
Continuous vs dynamic batching for AI inference

Apr 5, 2024 | baseten.co | Matt Howard |Philip Kiely |Pankaj Gupta |Phil CardyScarlet Howes

Batch inference is essential for serving LLMs and other generative models in production. If you only run one request at a time through a GPU, most of its capacity is sitting idle. Running multiple inputs through the model simultaneously uses more of the GPUâ€™s resources to massively increase the throughput of your model deployment. However, itâ€™s important to choose the right strategy for batch inference to make sure your model still performs well on other important metrics like latency.
80% of Your Data is Unstructured and Potentially Unprotected - 4 Strategies to Tackle This

Apr 4, 2024 | cdomagazine.tech | Matt Howard

When it comes to data-centric security, there are two priorities that every organization must contend with. The first is to protect structured data (information stored in rows and columns within databases) which typically makes up 10-20% of a data estate. The second is to protect unstructured data ( documents, emails, and everything else) which represents 80-90% of a data estate.
Using fractional H100 GPUs for efficient model serving

Mar 28, 2024 | baseten.co | Matt Howard |Vlad Shulman |Pankaj Gupta |Philip Kiely

NVIDIA H100 GPUs support Multi-Instance GPU (MIG), which lets us serve models on fractional GPUs. We can get two H100 MIG models serving instances per H100 GPU, each with about half of the power of a full GPU. Splitting H100 GPUs into two parts allows for more flexibility in hardware choice for model inference.