Structured, semi-structured, and unstructured data are the three categories of data sources that can be classified. Unstructured data account for approximately 80% of all global data, whereas structured data account for only 20%.
As models have become more sophisticated and pushbutton, AI teams have realized that focusing on data iteration is just as important, if not more so, for developing and deploying high-accuracy models successfully and efficiently. ML models have become increasingly complicated and opaque in recent years, necessitating significantly larger amounts of training data. In addition, data have evolved into a useful interface for working with subject matter experts and transforming their expertise into software. Finally, data-centric AI enables a higher level of model accuracy than was previously feasible using only model centric techniques.
Datasets are dynamic. New files and new versions of existing files enter the datasets at the ingestion stage. Additionally, extractors can evolve over time and generate new versions of raw data. As a result, datasets versioning is a cross-cutting concern across all stages of a datasets. Of course vanilla distributed file systems are not adequate for versioning-related operations. For example, simply storing all versions may be too costly for large datasets, and without a good version manager, just using filenames to track versions can be error-prone. In a datasets, for which there are usually many users, it is even more important to clearly maintain correct versions being used and evolving across different users. Furthermore, as the number of versions increases, efficiently and cost-effectively providing storage and retrieval of versions is going to be an important feature of a successful datasets system.
JZFS was born for the above problems:
A Git-like version control file system for data lineage & data collaboration.
https://github.com/GitDataAI/jzfs
12 months ago
1
Apply to attend LabWeek Field Building at Edge Esmeralda, a pop-up event city in Healdsburg, CA, from June 10-16. The decentralized conference is co-curated with Foresight Institute and will encourage conversations and collaboration to drive progress on four main tracks:
Subscribe to the LabWeek Field Building newsletter for updates: https://labweekfieldbuilding.substack.com/
1 year ago
0
Office Hours are 15-30 minute meetings that members can schedule with others in the Protocol Labs network.
Use this time to discuss some of the most pressing issues your organization is facing. Some common topics include:
When used effectively, Office Hours can be an extremely efficient way to get timely advice, expand your network, and uplevel by learning from leaders in the network.
Learn more:
Please continue to learn how to make Office Hours super effective, and share your learnings in the comments below.
1 year ago
1
Building a company can be a simultaneously emotional, difficult and rewarding journey fraught with challenges. Product dev, legal issues, growth hacking and even the most minute administrative tasks at the early stages of company formation and expansion can be fraught with roadblocks. But how can you tell if the challenges you're facing are normal or a sign that you should pivot?
After tracking pivots in web3, we’ve found that there are four recurring indicators that suggest a pivot may be in order, with some signals evident even before a company raises venture capital.
Indicator 1: The Passion is Gone
Imagine a founder has just won a hackathon, they’re getting flooded with both investor and user interest. The accolades feel amazing and they’re on a high. They decide to raise a round based on this idea. However, before embarking on this journey they should ask themself, are they really passionate about this product? Is it something that they can truly see themself committing the same level of enthusiasm to over the next 10 years?
One of the more surprising reasons that early-stage companies fail is a lack of passion. It’s difficult for anyone to sustain enthusiasm for the length of time that it takes to build and scale a company through its many challenges. However, passion is one of the most important ingredients in a startup’s success. Throughout the company lifecycle, founders have to be the number one hype person for their company. Their passion is:
A lack of passion for product makes every day an uphill battle, even more so if founders are also operating in a space that they’re not as familiar with and need to combat their own disinterest as they learn the ropes. If through introspection founders realize that they’re no longer passionate about what they’re building, they should pivot.
Indicator 2: Being Too Niche
Beware the niche trap which can be particularly prevalent in the crypto and blockchain space given how early the market itself is. Let’s say a founder is extremely passionate about the idea they’re going after, but they keep running into the same roadblock in conversations with investors who are concerned about the market. This signal isn’t always cut and dry, as it could simply stem from investor ignorance. However, founders should be able to clarify this easily for investors, using their own research to illustrate that the market is indeed valuable.
While it’s said that there’s riches in niches, a market that is too small quickly becomes a cage. Founders can avoid falling into this trap by sizing the market themselves before engaging with investors. They should build a TAM and a SOM and validate demand for the product via user interviews/surveys. If their TAM is in the low millions or users don’t show interest in the product, this is a sign that they’re potentially building a solution in search of a problem, and this search will likely be in vain. Trying to service a market that is too small or compressing will limit the company’s ability to reach significant scale as demand will naturally have a low ceiling.This is a sign to consider pivoting.
Indicator 3: Consistent User Acquisition Woes
Some founders reach the point where they have a functioning product that they’ve maintained consistent enthusiasm for and their market is large and attractive, but they’re having trouble on the user acquisition front. This could present itself in a myriad of ways:
Ultimately these scenarios are all signs that user demand isn’t there for the product and indicators that you should consider a pivot.
Indicator 4: Extreme Disruption
Finally we’ve seen companies pivot because of extreme disruption in their space. For example, many realized that OpenAI’s product releases would devour their entire market. Some who were well-capitalized and earlier in development saw the writing on the wall and decided to pivot immediately.
This can be the most difficult indicator to recognize, because most startups claim to be in the business of disruption themselves. However, while it’s one thing to try to disrupt a stagnant legacy business, it’s another to take on a new incumbent that is much better capitalized and further ahead both on R&D and company building. In the worst case scenario, founders risk taking on an established and highly innovative market incumbent whose greater capitalization and brand name awareness threatens any chance they have to carve out even a tiny piece of the market.
It’s important for founders to keep their finger on the pulse of the market. If they’re able to recognize events or trends that could affect their product early-on, this can allow them to be offensive instead of just defensive.
To sum up
It can be difficult to recognize the signs that a pivot is necessary in the day-to-day tumult of fighting fires at a startup. However, it’s important to regularly evaluate whether or not all the necessary elements for startup success are still in place. Founders should ask themselves the following:
If founders struggle to answer any of these questions in the affirmative, it’s an indication that they should potentially consider a pivot.
One More Thing
Founders of great companies have unique non-obvious insights into their markets which others may not recognize, so it’s still important for founders to pay attention to their own intuition. For instance, if founders are positive on all of these signals that doesn’t mean that they must commit to persevering either. There are a myriad of other factors that could cause founders to leave a company, including health issues, family life, legal challenges, etc. And in all cases when pivoting, if founders are unable to come up with a compelling idea that fulfills the former exigencies, they should consider returning capital to investors rather than creating a rebound company. Persevering through multiple pivots is usually the worst outcome both for capital efficiency and esprit de corps.
Comment your thoughts and questions below. If you’d like to chat further 1:1, you can find my Office Hours here.
1 year ago
1
Protosphere is the front page of the PL network. The name is a portmanteau of “Proto”, representing Protocol Labs, and “Sphere”, as in an object’s sphere of influence of planetary pull in space.
Protosphere is a place for members of the network to build out their sphere of influence. It is a great tool to share your thoughts directly with the network and get engage with other members via comments and groups (coming soon!)
Most posts and features in Protosphere are private and require you login with the same credentials you use for the PL Directory.
Home view:
Topics view: (second icon on the lower left side bar)
To make it most time-effective for you, it is recommended that you post articles or topics that you’ve already written about elsewhere, whether it’s a blog post or announcement email or Twitter thread. Pick topics that you would like to get direct engagement and feedback from members in the Network.
Some good examples are below:
If you have any questions about ProtoSphere, or come across any technical issues, please reach out immediately to spaceport-admin@protocol.ai.
1 year ago
0
This guide teaches you how to search for and connect immediately with PL network members using Office Hours. It also showcases cool ways other network members are using their Office Hour links.
Think of Office Hours like what your college professor used to host before midterms. Office Hours in the PL Directory give you direct access to scheduling a meeting on someone’s calendar. The link is usually a Calendly or Cal.com URL. Scheduling Office Hours with someone allows you to save time sending cold emails or Telegram messages that never get a reply.
Only members who are open to giving a select chunk of their time to Office Hours for the network have these links on their profile, so you can feel certain to get a response. Please be sure to state clearly the purpose of your meeting, show up on time, and be prepared.
You must be logged in to the Directory to see Office Hours. If you don’t know how to login, follow the instructions here.
**You can also use the Skills filter on the left-hand side to narrow down your search.
Check out Office Hours Guidelines and Tips to learn more!
If you’d also like to provide Office Hours for the network, follow the instructions below to add an Office Hours link to your profile.
1 year ago
0
The PL Directory is the main source of truth for network membership and your best way to stay connected and find other members in the PL network.
Please follow the instructions below to login to and update your profile. If you have any questions, please email spaceport-admin@protocol.ai.
Because your email is very closely tied to your Directory account, updating it requires an additional verification step. If you need to change your email for any reason, please follow the instructions below:
Make your profile more searchable by adding your professional skills. This is something that people can filter by when they search for members to connect with.
To update your skills, login to the Directory and follow these 3 easy steps:
You can increase the ability for folks within the directory to connect with you by updating your additional contact information and social handles.
The privacy settings in the directory will allow you to choose the social and contact information that you share with the network. If you haven’t added any yet, make sure you go to the Social tab under Profile first.
Share the contributions that you have made to projects with the network to allow members to understand your areas of expertise and help them find you.
If you are the lead of your team, and your team hasn’t been added yet, you should do so as soon as possible so your team gets included in network-wide events and communications.
*NOTE: You will need to be logged out of the directory when adding your team. To log out, click the dropdown arrow in the upper right corner, next to your profile picture, then choose “Logout.”
To add a new team, you need to complete 2 steps: 1) Add a Team to the Directory, and 2) Add it to your profile.
Your request will be reviewed by Spaceport, and once it is approved, you will receive an email notification that your team has been successfully added to the directory. This should take about 24 hours.
If you are a member of more than one team, you can click “+ Team and Role”. Then select your new team and enter your role. Don’t forget to click “Save Changes” when you are done.
Other members in your team can now do this as well. Please share the “Add a Team to your Profile” instructions with them.
If you come across any issues, please reach out to spaceport-admin@protocol.ai.
1 year ago
0
