You could be right, but it's a lot of work effort to do this port so I imagine it's not a trivial limitation where are few lines of dropping into racket code would do the trick. It gives me the sense that he's hit a much bigger or deeper[1] problem.
I've tried to start back into arc coding, a few times, but I immediately hit problems. I've found clojure's hash-maps/tables to be a fundamental need and arc tables to be far too limiting. I've considered forking arc an implementing them, but I can't find time. I'm too busy writing apps.
I like Clojure as a language, but I'm a bit overwhelmed with choice when it comes to web frameworks and the package managers you need to use those web frameworks (somewhat similar to Common Lisp), whereas with Racket and Arc there's just one place to start. Do you have any experience with Clojure web frameworks?
I'm always a bit scared of adding external dependencies to any project, but Rich Hickey has this great rant about semantic versioning where he argues that a new "major version" is essentially just a bad excuse for breaking existing stuff. I wonder if Clojure web libraries take those principles to heart or whether it's another left-pad incident waiting to happen?
note: it's a 2010 article so some of it's outdated, but the idea would be the same.
At that time everyone was racing to make more robust web frameworks. Many of them were from people doing the above stuff only adding features. However, shortly afterwards cljs was released and another slew of web frameworks came out as people embraced writing web apps client side. Then again, shortly afterwards, Facebooks React became the new thing and advanced the idea of further separating out the data content from the UI composing for rendering. At this point data models (i.e. big hash-maps) and syncing that data to the UI became the new norm. And even since then more advanced frameworks came out, such as Fulcro, that further extend the data modelling & syncing features (https://github.com/fulcrologic/fulcro).
Through out all this many web frameworks became abandon-ware and now it's really hard for a newbie to make sense of which one to use. In my opinion:
1. If you want to do what Arc does (server side page generation) then use Compojure + Hiccup.
2. If you want to write basic client side cljs code there are dom libraries like:
2a. If you want to write client side cljs code that takes advantage of React then use Reagent.
https://github.com/reagent-project/reagent (Much better more interesting that Domina/Dommy)
If you have a desire to enter the more advanced data-model-UI-syncing arena where I would probably use Fulcro (but haven't). Note that these advanced frameworks like Fulcro expect you to know much more about state management / data modelling and it could be a steep learning curve for some.
I've been developing cljs web apps for over 4 years. Over these years I've tried some of the frameworks, but I ended up writing my own as none of them could do what I needed.
Is that helpful?
All of that may seem like too much, but remember you really only need Compojure + Hiccup to be where Arc is at.
That Compojure example does look quite familiar and Arc-like, and looks like it can handle multipart post requests too[0].
I'd never heard of Fulcro before. Given what is often emphasised about Clojure, at first glance at the Fulcro docs I'm a bit surprised how often they mention state and mutations:
> The other very common case is this: You’ve loaded something from the server, and you’d like to use it as the basis for form fields. In this case the data is already normalized in your state database, and you’ll need to work on it via a mutation.[1]
Also, I'm too much into graceful degredation to ever go all out Cljs, unless it was for a phone app. But I find that it's often interesting to see how people do things in Clojure, even when not using that language, so I'll be giving those Fulcro videos a look.
> I'd never heard of Fulcro before. Given what is often emphasised about Clojure, at first glance at the Fulcro docs I'm a bit surprised how often they mention state and mutations:
Well things on the client side can be sometimes be mutable. No one gets around the fact the DOM is a mutable only object. But besides that, the Fulcro library has labelled one of their feature's a 'Mutation'. Which was probably a bad choice, but it has nothing to do with the immutability of the underlying cljs object that it uses for that "Mutation". You'll notice the example is using 'swap!'. That means it's modifying an atom; Where an atom is an interface to make changes to the immutable object it holds. So really 'swap!' takes the change request, constructs a new version the original thing held in the atom, with changes, then 'swap's it with the original item inside the atom. The original thing was never changed (no changes to existing slots in memory). Hence clojure's things are immutable, and they are in Fulcro too, accept when changing the DOM tree.
As for state that's mentioned all the time in Clojure :)
This is the Rich Hickey talk from 2016: https://www.youtube.com/watch?v=oyLBGkS5ICk. If you listen to it, it was made in an effort to adjust the community's trajectory. I too would like to hear if adjustments did happen.
Basically the way I interpreted it[1] is that "major version" is a meaningless concept. If you're making incompatible changes, rename the package. If that's a disincentive to making incompatible changes -- great!
> Rich Hickey pointed out last year that the convention of bumping the major version of a library to indicate incompatibility conveys no more actionable information than just changing the name of the library.
Yes, I guess Hickey is applying this idea of immutability not only to data structures but also to APIs and even databases[0] with the proprietary database service Datomic. Interestingly Datomic uses Datalog as query language, so it's straight forward to apply at least some of those ideas with the Racket Datalog package[1].
As of now I have a basic web forum working with all data storage done in Datalog (except for file uploads!). I'll post some code once the design is a bit more settled. It's still in the breaking-things-all-the-time phase.
What I find a bit tricky about Datalog is that relations are stored together with other facts, which in my mind feels a bit like storing code in a database, but maybe I just haven't wrapped my head around it yet.
Does anyone here have favourite articles or talks about logic programming?
> What I find a bit tricky about Datalog is that relations are stored together with other facts, which in my mind feels a bit like storing code in a database, but maybe I just haven't wrapped my head around it yet.
I can't speak for Datalog, but I've used Datomic.
If it's the same, then a 'fact' is comprised of an entity (the id), + an attribute, + a value.
The relationships are made by storing an entity id into the value slot of another fact. Thus the model is both flat (being a list of facts) and hierarchical (they can point to each other). It pretty much becomes a graph database. Is that what you mean?
> I can't speak for Datalog, but I've used Datomic.
Cool!
> The relationships are made by storing an entity id into the value slot of another fact. Thus the model is both flat (being a list of facts) and hierarchical (they can point to each other). It pretty much becomes a graph database. Is that what you mean?
Yes, exactly! What I worry about (because I'm fairly new to logic programming) is whether it could potentially be difficult to update a program where storage of business logic and storage of data aren't separated?
If we assume we have a Hacker News web app where we have a fact: One day Alice submits a story with the title "How to peel onions"; and we are thinking "Why on earth did she post that here?!?" So we add this relation to our code: A story is `irrelevant` if it has the word "onion" in the title. Then, the next day we get another fact: Now Bob has submitted a story called "How onion routing works". This new story by Bob then makes us reconsider our definition of `irrelevant`.
In a typical imperative program we'd just edit the code and redefine the `irrelevant` predicate, and it would take effect next time we run the program (or instantly if we enter it at a repl). But here in our logic program we store this `irrelevant` relation in our graph database, so even though we have removed it from our code, it is still sitting there in the database along with all the facts, outside the reach and responsibility of git, or whichever VCS we're using.
Yes, so my question is: How do you practically deal with changes to business logic in logic programming where data storage and relation storage is one and the same? Perhaps Datomic just avoids this issue somehow? I may also be missing or misunderstanding something.
Can't say I know what the options are since I don't know Datalog or the DB you're using, but is this reasonable?:
-----------------------------------------------------------
Entity | Attribute | Value
-----------------------------------------------------------
person-id-001 | name | Alice
person-id-001 | stories | [story-id-001, story-id-002...]
story-id-001 | headline | "How to peel onions"
irrelevant-word-001 | stories | [story-id-001, ...]
irrelevant-word-001 | word | onion
-----------------------------------------------------------
So if you decide that onion is no longer irrelevant then delete the entity 'irrelevant-word-001'. Which seems, at least to me, better than making code pushes.
So all of this assumes a few things:
- Your DB supports a cardinality of 'many' items in the value slot.
- Your query language can perform joins.
Of course none of this helps when someone changes a headline, but only full-text search DB's will help you do that.
So you would also need to remove the value 'irrelevant-word-001' from the above.
At least this is how I would do it in Datomic anyway.
What's interesting (at least to me) is that Datomic has a function called 'retractEntity' [1] which auto-magically removes all references of an entity in any value slot when you retract the entity. Man I love Datomic :)
> So if you decide that onion is no longer irrelevant then delete the entity 'irrelevant-word-001'. Which seems, at
least to me, better than making code pushes.
I can make sense of that when there's just one instance of this app running. Yet imagine the scenario where the web app has been published, and suddenly other people are running this web app. If the business logic is then changed, somehow I'd have to tell those people: "Oh btw, when you're running `git pull` next time, then you just also gotta run this query to retract some of the old relations from the database."
Definitely not a problem for me yet, but I can just smell it coming. I could add those retractions to the code, but they would have to stay there indefinitely, because it's not possible to tell if those retractions have taken place on everyone's databases yet.
Maybe I'm over-thinking this.
> What's interesting (at least to me) is that Datomic has a function called 'retractEntity'
Looks like the one called `~` in Racket's Datalog[0].
Is this irrelevant-word example a real feature you're building into the app or a contrived example to understand Racket DataLog DB use?
If it's a real feature, and I'm assuming it is. Then I'm also assuming that when a story is submitted you're parsing the title and adding the relationship to the current set of irrelevant-words that are stored.
So the question's are:
1. How are you going to remove the past relationships between stories and an irrelevant word that's getting removed? (looks like we've answered this).
2. How are you going to make sure past stories gain the relationship to newly added irrelevant words?
3. How are you going to handle title changes.
After you get handle on these then what you really need to do provide an interface, from within the apps admin tools, to trigger the noted functionality. This way the business logic is in the app and it's modifying the data.
> Is this irrelevant-word example a real feature you're building into the app or a contrived example to understand Racket DataLog DB use?
It's a simplified and slightly contrived illustration of an issue in this pre-alpha code I haven't published yet, perhaps just because I haven't thought of a name for the project yet. But yes, let's assume it's a real feature for now.
> 3. How are you going to handle title changes.
I like Hickey's idea of accretion of data - with a timestamp! - and not forgetting previous facts. I think he's talking about it in The Database as a Value[0]. So, a story could have a few different titles in the database, and the newest one is the one you get to see.
The thing is, it's easy to add a timestamp to facts as a way of not considering old facts without forgetting them, but not to relations. For example in Racket Datalog[1]:
(! (voted "i4cu" 'up 134))
can easily be get a timestamp:
(! (voted "i4cu" 'up 134 1542151773))
but that would not really make sense in a relation like
(! (root A B) (ancestor A B) (parent null A))
So, I don't feel there's a need for ever retracting facts, because timestamps solve that. (Even when deleting something, you could just add the fact that is has been deleted.) But with relations (a.k.a. business logic) I think I will need to retract things, which is tricky because this logic is not only present in the code but also in the database. This was the problem I was asking about.
> 2. How are you going to make sure past stories gain the relationship to newly added irrelevant words?
Datalog queries reflect the current set of facts and relations, so the possible irrelevance of a story would not be stored anywhere, so it wouldn't need to be updated.
> 1. How are you going to remove the past relationships between stories and an irrelevant word that's getting removed?
Some as above. For example, in a place oriented program a story object could have a boolean attribute `irrelevant?`, whereas I'm sending a query every time this value is needed, so no stale `irrelevance` attributes are stored anywhere.
> After you get handle on these then what you really need to do provide an interface, from within the apps admin tools, to trigger the noted functionality. This way the business logic is in the app and it's modifying the data.
Yes, that is kind of there already, as in having functionality for changing titles. The `irrelevant` functionality is not there now.
Ok, I think I just need to work on getting this code publishable, because it might be easier to discuss tangible examples.
Yeah, Datomic doesn't expect the relationships to be stored in the DB. It stores a bunch of indexes for you and it has a great query language, but that's it.
So where will you're data be? In a local data structure? I read your racket Datalog link, but it doesn't show any details for the database side (i.e. durability etc.) even though it's labelled a database.
Also, I'm curious what made you choose a graph db. It seems like you're inheriting a lot of complexity and I'm wondering what the benefit is over a more traditional sql or nosql db.
> So where will you're data be? In a local data structure?
Yes, I think. The database just stored in memory but it can be serialized and saved to the disk using `write-theory`[0] and loaded `read-theory`. That is what I'm doing for now, and it's a very naive and inefficient to do a full database dump rather than just appending new data, and I presume it's particularly in this area where Datomic is way more optimised and well thought out.
> Also, I'm curious what made you choose a graph db. It seems like you're inheriting a lot of complexity and I'm wondering what the benefit is over a more traditional sql or nosql db.
Well, I did the initial work on the web app: creating user accounts, adding posts and replies, and then I got to data storage. Initially I did a News-style flat-file database, just saving data as lists in files, that are then loaded into memory when the program starts. It mostly worked but also felt a bit complicated, and I thought that perhaps I should just use a proper database?
What I like about news.arc is that you can just launch it without any configuration, so MySQL and PostGreSQL were out of the question, and I started reading a bit about SQLite. But I've also had this fascination with logic programming, from what people are posting here[1][2], and from reading a bit of The Reasoned Schemer, and I watched some of those Rich Hickey talks again, where he talks about Datalog, which happens to be available for Racket.
There are just some things that are incredibly simple in declarative/logic programming. For example, if you have facts about stories being `parent` of their replies, then it's simple to just define the `ancestor` relation, and when you have the `ancestor` relation, you automagically get `descendants` without having to write any code, because it's just the inverse of `ancestor`:
(! (:- (ancestor A B)
(parent A B)))
(! (:- (ancestor A B)
(parent A C)
(ancestor C B)))
But, I've also bumped into some questions - more practical than theorical - and that is why it's interested hearing about your experience with Datomic, and why I'm asking here.
So, SQLite is still on the table. I'm not too familiar with NoSQL, but my impression is that they are all about speed and scalability of data storage. I haven't used MongoDB but isn't it essentially just like storing JSON in a file, except faster? It would be interesting if any of those could be used in conjunction with Datalog though, if don't add too complexity for the sake of increased speed.
Datomic uses DataLog as part of its query language, but that's pretty much where the comparison should end. Things like "treating the database as a value", and features such as data accretion that Rich talks about have nothing to do with DataLog. They're features of Datomic. So for example when you mention never retracting data, well your data size is going to continuously grow unless you write your own data management layer on top. Datomic, on the other hand, does this for you. When you want to query the database over time, then you're going to need to store time intervals for all of your data and incorporate that into each query. Where as in Datomic (which has a time log) you can pass in the DB itself as a value (with an associated time interval) and Datomic will make sure your queries are working against the dataset that accounts for the time interval.
I'm pointing this out because it seems to me that you're doing (or are going to be doing) a lot of work that may not be worth it for what you're trying to accomplish.
Nosql
> I'm not too familiar with NoSQL, but my impression is that they are all about speed and scalability of data storage.
Yes and No. Often speed can be a feature Nosql dbs advertise, but really, for me anyway, it's about flexibility and ease of use. Traditional RDBMS, for example, require creating schemas. Many Nosql databases don't require a schema at all which makes it easier to use and more flexible to change. Nosql's are often a key-value store so it can be really easy to take a hash-map or table of data from your code and just dump it into an nosql datastore and be able to query it.
My personal favourite is Redis and it might be worth considering for your app.
You can:
- store a value under a key [1]
- store table data [2]
- store values in a set [3] (which allows intersection/difference queries)
- store values in a sorted-set [4] (which allows you query by some numerical value like timestamp)
- use it to manage relationships [5]
The reasons I mention Redis is that the HN app is very well suited to it. HN only keeps 'x' amount of data in memory. And in Redis the data lives in memory. Also Redis allows you to set expiry times on data for auto eviction [6]. And Redis also supports ordered lists [7] which can make it useful for lisp based languages.
However it's not embedded. And if that's a requirement I'd almost suggest you move away from Racket and adopt a language that has more options for embeddable databases. I guess if you're willing to roll your own (and it looks like you may be) then that's awesome too.
But in case you decide otherwise... The library I use is Redis Carmine [8], but there are Racket clients [9].
> Datomic uses DataLog as part of its query language, but that's pretty much where the comparison should end. Things like "treating the database as a value", and features such as data accretion that Rich talks about have nothing to do with DataLog.
I'm not sure I totally agree with this. I think that apart from talking about the design of Datomic, he also has a more general point against what he calls PLOP (PLace Oriented Programming), which Datalog does address.
For example in plain Racket a value is lost if something else is put in its place:
Hickey is also mentioning how git doesn't do PLOP in that it doesn't throw out your commit history (without you asking it to do so).
> The reasons I mention Redis is that the HN app is very well suited to it. HN only keeps 'x' amount of data in memory. And in Redis the data lives in memory. Also Redis allows you to set expiry times on data for auto eviction [6].
Interesting. Just checked news.arc, and yes `initload*` is set to 15000. Interesting idea from Redis with expiry times. I'll check it out. I hadn't considered the scenario of storing text enough to max out on memory, because it would probably be premature optimisation, but good to keep in mind. I'd like to give Redis/Rackdis a try; thanks for the suggestion. I've been hosting an Etherpad Lite instance, and Redis was painless to setup.
> I'm pointing this out because it seems to me that you're doing (or are going to be doing) a lot of work that may not be worth it for what you're trying to accomplish.
Yes, my priorities here are definitely to make the code as brief and simple as possible, and to not have to do to much work. With plain Datalog it's very little work to timestamp a fact, and it's also kind of necessary, e.g. to figure out which fact is most recent, when previous facts are not removed. I'm just trying to get the gist of Hickey's ideas here.
> PLOP (PLace Oriented Programming), which Datalog does address.
Yeah, I was thinking more along the lines that Datomic has built-in functionality to address the caching, cache eviction, and indexing that goes along with all that data accumulation. But you're correct, DataLog does accumulate facts.
> Interesting. Just checked news.arc, and yes `initload*` is set to 15000.
I did the same thing, about 6 or 7 years ago, that you're doing now. I ported HN to Clojure (which is actually how I learned Clojure). If memory serves me correctly when I was doing the work I realized I needed a real DB if I wanted to support load balancing. i.e. I needed to centralize the data for the authentication and fnid session info. I think Arc calls them fnids... You probably know better than I do now, but Arc has all this code to expire these session fnids and so, for me, Redis was just a good fit for that task.
Anyways, I'll be sure to take a look at the final result of your work.
> I did the same thing, about 6 or 7 years ago, that you're doing now. I ported HN to Clojure (which is actually how I learned Clojure).
Cool!
> I needed to centralize the data for the authentication and fnid session info. I think Arc calls them fnids... You probably know better than I do now, but Arc has all this code to expire these session fnids and so, for me, Redis was just a good fit for that task.
The Racket web server is quite "batteries included" and comes with these different managers[0] for dealing with expiration of sessions/continuations, such as the LRU manager:
> The memory limit is set to `memory-threshold` bytes. Continuations start with 24 life points. Life points are deducted at the rate of one every 10 minutes, or one every 5 seconds when the memory limit is exceeded. Hence the maximum life time for a continuation is 4 hours, and the minimum is 2 minutes.
> If the load on the server spikes—as indicated by memory usage—the server will quickly expire continuations, until the memory is back under control. If the load stays low, it will still efficiently expire old continuations.
When I was referring to load balancing and centralizing the data I was referring to many web servers sharing a centralized/external source for auth/session data.
I'm unfamiliar with racket's web server 'servlets'. The docs are little unclear (at least to me). Can these servlets live on a separate server so that the data can be shared between web servers? I'm guessing that was/is not a requirement for you, but I'm just interested in knowing if that's how it can work.
Uh oh, you're getting me interested in Racket now. I can't have that... I have too many projects :)
edit: I guess at the end of the day these servlets are web-servers right, so you can, even if you have to do it over http and build an api.
> Can these servlets live on a separate server so that the data can be shared between web servers?
Probably. I assume that serializable continuations[0] from stateless servlets can just be stored wherever, like in Redis or something, instead of in the memory of one server.
> I ported HN to Clojure
If that is something you have published, it'd be fun to see, whether it's finished or not.
> Uh oh, you're getting me interested in Racket now.
My impression is that Clojure is faster, less verbose partly due to clever syntax and provides more immutable data structures than Racket. But when it comes to documentation and error messages, I find Racket more coherent and comprehensible.
Say, if I wanted to connect to a SQL databse, with Racket I'd use the DB module[1], end of discussion. But with Clojure there's Korma, ClojureQL, Persist, HoneySQL, Yesql, a JDBC wrapper from Clojure contrib, SQLingvo, oj, Suricatta, aggregate, Hyperion, HugSQL, and probably a few more[2][3]. That multitude of libraries with similar purpose may be useful in some cases, sure, but also potentially a bit overwhelming for beginners, so I guess that's why I found it easier to get started with Racket.
> If that is something you have published, it'd be fun to see, whether it's finished or not.
I actually tried to look it out the other day during this conv, but it's buried somewhere unavailable right now. If I find/get to it I'll post.
> That multitude of libraries with similar purpose may be useful in some cases, sure, but also potentially a bit overwhelming for beginners, so I guess that's why I found it easier to get started with Racket.
Agreed. Navigating the volume libraries and the options available is a real pain in the beginning, but once you get past that, then it's not bad at all. At the same time, take a look at the quality of Clojure's Redis Carmine Library vs. Racket's Redis Libraries. Miles apart.
You'll need to elaborate more, apologies in advance to your thumbs. Some of the features you describe I don't understand (ability to read, maybe others). Others seem to already be in Arc (library support, iteration, destructuring bind; [_ 'user] doesn't seem much worse than :user). Nil values feels like the only definitely missing feature, and it feels more like a fork in the road with different trade-offs rather than one alternative being always superior to the other.
Reading that output is beyond a headache. It should be against the law! But more importantly, when writing code/troubleshooting you can't copy the evaluated output make a modification and then call the function again. I know you can bind it and apply functions to modify it, but often that's a real hassle compared to copy/paste.
Clojure has DOZEN's of helper functions (and important traversal functions) that don't exist in Arc. Yeah I could write them, but I still have to. I was highlighting the ease with 'separate', but here's even 'deep-merge'....
(defn deep-merge
"Recursively merges maps. If keys are not maps, the last value wins."
[& vals]
(if (every? map? vals)
(apply merge-with deep-merge vals)
(last vals)))
> (deep-merge players {:joe {:weapon "LongBow"} :jane {:weapon "Lance"}})
It's not even these sample cases that expose the issues. It's about working with REAL data. I have many files containing hash-maps that are 15 levels deep. If I were to attempt pretty much anything, but the most simple cases, in Arc I would hit problem after problem after problem. Which I did. So fine... really I should use a DB, right? ok let me just grab that DB library in Arc....
-- Nil values --
I'd even be fine with any falsey values like boolean false. But Arc will not even store an #f value. To suggest I can't pass in REAL data with falsey values is really limiting. I can't pass in an options table argument. For every operation I would have to add more data or nuanced data or nuanced operations to support "this value was set, but it's set to 'NO' I don't want that option.".
Firstly, Warren Buffet's take on Bitcoin is the same take that he has on Gold. And do you believe Warren Buffet doesn't understand Gold? Don't you think Gold investors write articles on how Warren doesn't understand Gold? Just because someone doesn't buy into something doesn't mean they don't understand it. Which brings me to my second point: Blockchain is not Bitcoin.
From a technology perspective Blockchain will increasingly become more important. From an investment perspective I'm with Warren Buffet - Bitcoin is shit.
It's important to understand that Blockchain as a tool can create technology which will have utility (and thus value), but Bitcoin has limited utility and no intrinsic value, in just the same way Gold has no utility nor any intrinsic value. Just because Bitcoin utilizes Blockchain doesn't mean Bitcoin inherits that value as a product (or in this case: currency).
Edit: BTW - the rest of the article was great... I just couldn't get over the Buffet/Investment part (which to be fair is pretty much the opening statement for the article).
Thank you!
I don't think there's any example of a systems which fulfills the desired properties of a Blockchain without a native cryptocurrency.
I'm skeptical of the notion of intrinsic worth. I think things have value because humans value them.
Gold has a lot of utility, you probably use it frequently without knowing it.
https://www.popularmechanics.com/science/a19670/refining-gol...
I think it's more accurate to say that Blockchain uses Bitcoin, since Blockchain was the solution necessary to create Bitcoin(speaking of course before the advent of derivative cryptocurrencies+blockchains), but really they're codependent.
You need bitcoin to write to Bitcoin's blockchain. That's very useful.
> You need bitcoin to write to Bitcoin's blockchain.
Right so, Blockchain is a methodology for a cryptographic system of record with transaction capability, and doesn't have to be used for currency, even though that's how everyone is using it. When you simplify it down to a 'system of record' then other use cases become new opportunities. I don't currently have useful ideas outside of currency, but my point was that Buffet doesn't need to understand the details of the methodology (bottom up) to establish an understanding of Bitcoin for investment purposes.
Even as a currency it is valuable provided that it gets tied to something tangible/meaningful. Personally I'm interested in ethereum because of its smart contracts and dapps.
> Gold has a lot of utility,
I'm aware that gold has some utility, so you are correct that I shouldn't haven't stated 'no utility', but gold's utility is not capable of justifying the price on the market. If gold was not being held as asset for the sake of being held as an asset, based on its rarity, then its market value would plunge to the near bottom.
> I'm skeptical of the notion of intrinsic worth. I think things have value because humans value them.
Right so, Gold has value because humans value it, but its value on the market is not tied to its intrinsic worth it's tied to its rarity alone. And that's the point of highlighting intrinsic worth over a human valuation. The value stability/durability is tied to its intrinsic worth.
The overall point is that Buffet is a value investor that's known to use intrinsic worth as the value measure. This is why Buffet hates both gold and Bitcoin. So let's not assume that he doesn't understand. Bitcoin's value is not stable because it's not tied to anything tangible. Even $ are just notes, but they are backed by a gov't which is why they have intrinsic value and therefore much more stability (or as much stability as the gov't/country provides).
note: made some edits to support correct word use.
I've found the 'hook' mechanism to be really useful. I use the same thing in an app I'm writing (in cljs) which auto creates web applications via data schematics.
When my app compiles a new app there's a little routine that adds/removes pre-defined function calls from within a table. So then when cljs compiles the client side code (which performs dead code elimination) the functions that are hooked into the table get added, but the others are eliminated. The cljs dead code elimination process couldn't do this alone; it required being able to hook/unhook code depending upon the app schematic.
Anyways, just wanted add to your point on how useful this feature can be.
I don't believe so, but it depends on what relationships and queries you have in mind.
I think you just need to maintain an index of story ids for each tag then alter the page list code to check each story against chosen tag indexes in order to apply custom filters. And you will also need to bypass the page caching operations for these filtered pages (just given the number of combinations that are possible).
Obviously db capabilities would make it (and everything else) much better, but it's not a showstopper.
I like the invitation tree concept. I plan to do that with one of my apps, but I think I would have to add that feature well after I obtain a decent user base.
In order for the News app to have a CSP and be strict about it, you would need to:
1. Remove the inline js. This means the votelink code (votejs) needs to be moved from news.arc and put into an external file (news.js?) that is linked to as a file within the header.
2. The inline onclicks need to change. The onclick values have to be actual function pointers not strings and given Arc has no built in js functionality that likely means removing them completely. Instead you will need to have a js call in the new 'news.js' file that does document.addEventListener with the 'DOMContentLoaded' argument along with a function that finds all the relevant doms for a given page and adds listeners to each that will trigger the votejs code.
3. srv.arc needs the addition of a Content-Security-Policy header for server ops (with the appropriate settings).
4. All inline style attributes need to be removed and changes to news.css or news.js will need to be made in order to compensate. i.e. stuff like this:
(div style "margin-top:1px;margin-bottom:0px")
edit #1. note that adding the hash code referred to (or even the 'nonce' option) is a hack intended to provide short term relief to production environments until proper changes can be implemented.
edit #2. regarding point 4 I believe (but not absolutely sure of) that all the inline font, color, font-size tags are a problem too. i.e. It's any tagged string value that will be interpreted by the browsers css engine. If anyone can confirm this, please do. Either way, none of that stuff is HTML 5 compliant and probably should be removed anyway.
>srv.arc needs the addition of a Content-Security-Policy header for server ops (with the appropriate settings).
What's needed is the ability to pass custom headers from the application to srv.arc (or maybe app.arc) since CSP headers would be application specific. Unfortunately, unless I'm wrong, it looks like header generation is baked into srv.arc.
> All inline style attributes need to be removed and changes to news.css or news.js will need to be made in order to compensate. i.e. stuff like this:
A lot of that can be removed altogether by removing the table layout and just using a basic grid. There's no reason the forum has to be pixel-perfect. This would have the added benefit of letting us get rid of a lot of hacky one-off table macros in html.arc.
> What's needed is the ability to pass custom headers from the application to srv.arc
There is the possibility of just putting the CSP into a meta tag within the page header, but I didn't suggest that because not all CSP options are available when using the meta tag.
I think you're right in that being able to dynamically add headers is the right way to go. When I moved from arc to clojure I did this by implementing something like arc templates [1] and used them to pass attributes through to the server ops. I ended up with a 'defop' like call that took an options hash-map argument (i.e. a template instance) which then generated the headers dynamically (with built in sane defaults).
>Yeah the whole thing should get HTML5-alived. CSS, JS and web-standards have evolved significantly since the app was originally written.
Sorry to necro this but I'm hoping to have an update to the HTML and CSS finished soon.
I've consolidated the page templates to a single macro for the entire document, removed the tables and inline css, and got the threads working with unordered lists. I think the result might be as close as we can get in Arc to what you're describing (which is the way most frameworks operate - a single base template) without a massive rewrite of src.arc and html.arc as well.
pg: "Arc embodies a similarly unPC attitude to HTML. The predefined libraries just do everything with tables. Why? Because Arc is tuned for exploratory programming, and the W3C-approved way of doing things represents the opposite spirit.
[...]
Tables are the lists of html. The W3C doesn't like you to use tables to do more than display tabular data because then it's unclear what a table cell means. But this sort of ambiguity is not always an error. It might be an accurate reflection of the programmer's state of mind. [...]
Good cleanness is a response to constraints imposed by the problem. Bad cleanness is a response to constraints imposed from outside-- by regulations, or the expectations of powerful organizations."
Personally, I think using semantic HTML isn't that big a deal to implement, and it seems to have practical benefits in terms of accessibility. It isn't just something the W3C is trying to impose on people arbitrarily.
And yet, the built-in features of HTML and CSS have a tendency of being very arbitrary. If you want a text box, you can have it; if you want a set of radio buttons, you can have it; if you want a set of radio buttons where one of them says "Other (please specify)" and has an associated text box, you suddenly have a significant amount of code to write. If you want to style the first letter of a paragraph, you can use a ::first-letter CSS selector... but if you want to style letters other than the first, you need to wrap them in explicit HTML elements, which unfortunately has other effects you might not want (like causing screen readers to treat each letter like it's a separate word).
Sometimes, there isn't a workaround. For instance, pages have titles, which appear in the top of the browser window. Have you ever seen a page with a title and a subtitle displayed just under it? I haven't, and short of finding a security hole that makes the browser execute arbitrary code, it seems pretty clear there isn't a way to do this.
Sometimes, there is a conceivable workaround, but it requires something like building your own text layout engine from scratch and then wrangling with a lot of obscure Unicode scripts, screen-reader troubles, text selection support, etc. There are some in-browser text editors which have to make this kind of effort just to achieve syntax highlighting.
And sometimes, there's a workaround that's a lot like building your own substantial subsystem of the browser, but it's actually fairly reasonable to do in a pinch. Like, there are a bunch of front-end frameworks for writing reactive UIs. They take in something that's pretty similar to DOM nodes (sometimes even obtained by parsing DOM nodes that aren't meant to be displayed to the user), they generate actual DOM nodes that are similar to those, and they modify those generated DOM nodes on the fly as the application state changes. In certain respects, these frameworks can save a lot of work by taking advantage of the underlying features of HTML... and in certain respects, there's extra work involved in inverting HTML's abstractions to get them to support this new indirect interaction style they weren't designed for.
pg: "But the advantage of a rewritable language is more than that it lets programmers fix your mistakes. I think the best programmers tend to work by rewriting whatever language they're using. So even the perfect language, if there is such a thing, would be very rewritable. In fact, if I had to guess, I think the perfect language might be whichever one was most rewritable."
How much code does it take for someone to implement their own "rewritten" variation of some HTML or CSS feature? Well, if they can't achieve their goal without writing their own text layout engine or their own virtual DOM framework, quite a lot.
This is important to Arc because it makes the program longer.
Arc is a language designed incrementally by starting with something Lispy and then making whatever changes will shorten Arc programs. News is a program that was written to put Arc to the test; the shorter its code is, the better Arc is doing.
If News used HTML or CSS features in very picky ways, it wouldn't be a good test: As soon as News's needs strayed slightly from the HTML and CSS features that browsers had built-in, then a heap of code would need to be written to make up the difference. When a slight discrepancy in the way a measuring instrument is consulted results in a large discrepancy in the measurement, it's not a very reliable measuring instrument.
So it seems to me that of all possible programs, News was pursued because it could get by on using HTML features in non-picky ways, pretty much in the ways they were already designed to be used. The use of HTML tables and transparent gif spacers was a well-known and once-popular, if dated, technique for achieving layout that was consistent across browsers, so pg built abstractions on that technique for News.
All that being said, I personally think it's a great improvement for News to use semantic HTML tags instead of tables, and I don't think this change really does that much to the size of the codebase (does it?). I just figure these pg writings are interesting in this context.
---
This has me thinking about html.arc....
The way html.arc is designed involves a lot of special-casing of specific HTML tags and attributes. It's almost like a full go-between layer abstracting HTML from Arc, which suggests that with some ambitious modifications to html.arc, it could turn into a DSL that compiles to HTML in a more indirect way (perhaps performing nonlocal transformations to implement things like footnotes or column breaks). This would potentially be a good place to hone the design of the HTML built-ins so that they're more abstraction-friendly and more "rewritable" as far as Arc code is concerned.
This has a lot in common with those front-end frameworks I mentioned. They abstract over and extend the features of HTML, and in doing so, they tend to make HTML's built-ins "rewritable" by exposing a new way for programmers to define their own extensions of the same kind.
(ノಠ益ಠ)ノ彡┻━┻ NO PG LISTS ARE ALREADY THE LISTS OF HTML
... sorry. Don't know what came over me there.
>and I don't think this change really does that much to the size of the codebase (does it?).
Most of it is the result of moving existing code around, so I think it comes out about even. I don't know how much of a performance issue macro expansion is but there is less of it in the new code, and the HTML itself should be simpler without tables.
>This has me thinking about html.arc....
Racket has its own xml/html library and there is an sml.arc which I haven't played with yet that seems like it might be capable. html.arc seems to do both too much and too little... the attributes blacklist makes it difficult to have modern features like data attributes, and the more macros there are, the more polluted the global namespace becomes.
I've sometimes thought it would be nice if Arc supported css and xml grammar natively, but I have no idea what it would take to actually support that. And I'm probably the only person here who wants to just write html and css directly, rather than use s-expressions or concatenating strings.
> Tables are the lists of html. The W3C doesn't like you to use tables to do more than display tabular data because then it's unclear what a table cell means. But this sort of ambiguity is not always an error. It might be an accurate reflection of the programmer's state of mind. [...]
Note that it's traditional html tables pg is referencing. I could be wrong, but my understanding is that the 'display' properties options: 'table', 'table-row', 'table-cell' etc, were not well supported (in IE particularly) or even existent at the time pg wrote the code. So he may not believe the same now.
> The predefined libraries just do everything with tables. Why? Because Arc is tuned for exploratory programming, and the W3C-approved way of doing things represents the opposite spirit.
I don't agree. My understanding is that traditional tables are ridged, which is why they suggest you only put data into it. They are highlighting that it's, generally, not suitable for other things. Divs allow for more flexible manipulation. For example, I can create a table and then decide to break out of the table somewhere in the middle of it's content to render some component. Or I can make a table and morph it into something different by only changing it's properties via css/js.
But pg was not writing web apps. He was writing web pages and then calling a new page for, pretty much, ANY change. So from that perspective (where you can macro away server side) you can see why pg would just put it all in a table and highlight how using arc macros will let you do more compose-able things.
Strict CSP settings are a form of whitelisting what js, css etc, is valid thus protecting from injection. Inline code for both js and css can't be whitelisted like header items can be so they will fail (unless you use the hash code hack mentioned for js).
"By controlling a little bit of text in the victim domain, the attacker can inject what appears to be a valid CSS string. It does not matter what proceeds this CSS string: HTML, binary data, JSON, XML. The CSS parser will ruthlessly hunt down any CSS constructs within whatever blob is pulled from the victim's domain...."
"A policy needs to include a default-src or script-src directive to prevent inline scripts from running, as well as blocking the use of eval() . A policy needs to include a default-src or style-src directive to restrict inline styles from being applied from a <style> element or a style attribute."
So it's just the 'style' attribute people worry about and strict CSP manages.
Thanks for all those links. Not sure I "get" the browsers of today. Not sure I can be bothered manually adding hash-codes for inline JS (Maybe doable from Arc, but sounds hacky).
It will be difficult to deal with some styling functions from Arc, like `grayrange` that's greying out comments with negative score. Perhaps JS is more suitable?
I wonder in which file the CSP would need to be implemented in Arc, or whether it's easier to set them in an Nginx config.
That's because it is a hack (as mentioned in my original comment edit#1).
My comments are only intended provide whatever help I can towards the original posting context which suggested a strict CSP criteria.
None of these things have to be done. It's up to you to decide, so really the question becomes what are you doing it for? Are you building a news site for a community of a few thousand people in a niche group? or are you making a news app that others can buy into for their own product/uses? The latter would make me want to ensure it's CSP capable, while the former - not so much.
> It will be difficult to deal with some styling functions from Arc, like 'grayrange'...
I would just create 10 or 20 or whatever number of css entries that act as a segmented gradient (call them .color-reduct1 to .color-reduct10) then create a server side function that takes the output value of grayrange and picks one the css entries. Then add that class to the html element and you're good to go. It's not a perfect gradient but it would be enough that I doubt it would make any noticeable difference.
Js is also an option, but then you have to store and pass the score into the js calculation which requires much more work then the above solution. Plus it forces you to expose the score (which HN no longer does)
> I wonder in which file the CSP would need to be implemented in Arc, or whether it's easier to set them in an Nginx config.
If you want to make code that's generic and useable by others then it needs to be in arc (not everyone will use Nginx). I suggested using arc templates [1] already and I still think this is the right way go. Establish the base template definition in srv.arc and then each app can modify that base template from their app file. Additionally allowing defop to optionally pass in over-rides will make it dynamic if you need that variance.
I'm sure there are dozen ways to do it, but that's my suggestion anyway.
> Not sure I can be bothered manually adding hash-codes for inline JS (Maybe doable from Arc, but sounds hacky).
No one bothers, everyone moves all of their JS to an external file (which is what i'm working on now) or they just don't bother with CSP headers at all.
>It will be difficult to deal with some styling functions from Arc, like `grayrange` that's greying out comments with negative score. Perhaps JS is more suitable?
The score for each comment could be added as a data attribute and JS could apply the style based on that. Offloading that to JS might make the forum more responsive.
...as well as, maybe, having markdown done entirely in JS, but that's for the future.
[edit] ... as well as maybe thread folding with JS and localstorage.
The reason why there's an 's' on 'withs' is that it's short for "with sequentially". Comparatively 'with' doesn't evaluate the bindings sequentially. So, for example, your 'val 0' is probably not being bound before the others are.
arc> (help get)
[fn] (get i)
Returns a function to pass 'i' to its input.
Useful in higher-order functions, or to index into lists, strings, tables, etc.
Examples:
arc> (get.2 '(1 2 3 4))
3
arc> (get!b (obj a 10 b 20))
20
arc> (get.9 sqrt)
3
arc> (map get.2
'((a b c)
(1 2 3)
(p q r)))
(c 3 r)
nil
arc> (help set)
[mac] (set . args)
Sets each place in 'args' to t.
nil
These are the functions you end up calling because your dispatch can't see the earlier get and set bindings.
The macro definition of with creates a function with all the names as inputs and the body of the with as the body of the function. The newly created function is called with the definitions of each name, which are effectively in independent namespaces.
The withs definition, however; recursively calls itself so that each succeeding name sees the definitions of previous names.
I believe the difference is historically due to higher speed of with. In modern programming it probably makes sense to use withs everywhere and only change to with in places where optimization is necessary.
I actually tend to the opposite: use with everywhere unless I need withs. The reason isn't performance. It tends to make code more elegant to not rely on the order in which things are defined. And when I'm reading code, with gives me the warm fuzzies that the code is going to be cleaner. When I see withs I slow down to look at the dependencies between the bindings.
Similarly to this, when I'm writing Java, I use `final`^1 everywhere I can. It's nice to be able to know that anywhere later where the variable declared final is in scope, it will have the same value as at the point it's set. I don't need to look through any code to see if it's rebound; I know it hasn't been.
[1] "final" is kind of like "const", if I understand `const` right. `final int x = 3;` means that it is an error to later have the line of code `x = 4;`.
Under knarks "Why the fork?" section hjek links to the 'ethical repository'[1]. Since notabug.com is only for open source projects then the repo's will likely be considered 'free software' which I believe makes it grade A in 'ethical repository' terms.
Personally I find the term 'ethical repository' offensive. It insinuates that non-free software is unethical when the majority of non-free software has no nefarious intent or code. Not exactly the greatest sales pitch in my book.
Yes, that's what I assumed as well; I just thought it would be an interesting discussion, and was wondering if there might not be other reasons for choosing that particular repository.
I suppose for me, the distinction between an 'open source' repository (Gitlab) and an 'ethical(?)' repository wasn't a very important one, so I was curious for the motivation behind it.
I think free (or "open source") and ethical mean the same in most cases.
Exceptions might include something like Facebook, which is technically somehow usable w/o non-free JS when using their basic mobile web page, but where the company is still engaging in other unethical activities, like selling user data to sway elections.
Or something like Amazon, where you might possibly be able to buy something w/o non-free JS (haven't checked), but where the treatment of their employees is unacceptable.
But, I think, when we're talking git hosting sites, there's no difference?
But FSF considers Gitlab ethical enough for hosting GNU packages[0].
As I understand it - the 'Open Source' movement concerns itself with improving the software by making the code openly accessible, where as the 'Free Software' movement concerns itself with a fighting for users rights (i.e. having the freedom to access, modify and distribute the code in a manner that empowers the user).
And so, an 'Open Source' repository holds code that is openly accessible for the purpose of improving the software. Where as an 'Ethical Repository' holds code that is graded by its' ability to guarantee users rights according to a specific set of morals (established by free software foundation). It so happens that open source repos tend to align well the ethics associated with free-software, but they should not be mistaken for each other. As an example to illustrate: If a repo SaaS were built for open source code, but restricted users from a certain country it wouldn't rank high in ethical repository grading. This is because while having the code openly accessible leans towards a Grade A rating (excellent), the restricting some users part puts it at a Grade F rating (unacceptable).
"Despite initially accepting it,[31] Richard Stallman of the FSF now flatly opposes the term "Open Source" being applied to what they refer to as "free software". Although he agrees that the two terms describe "almost the same category of software", Stallman considers equating the terms incorrect and misleading.[32] Stallman also opposes the professed pragmatism of the Open Source Initiative, as he fears that the free software ideals of freedom and community are threatened by compromising on the FSF's idealistic standards for software freedom.[33] The FSF considers free software to be a subset of open-source software, and Richard Stallman explained that DRM software, for example, can be developed as open source, despite that it does not give its users freedom (it restricts them), and thus doesn't qualify as free software.[34]"
> It insinuates that non-free software is unethical when the majority of non-free software has no nefarious intent or code.
The term can also be used by people who consider it unethical to even give programmers the possibility to hide nefarious code from users, regardless whether they actually do or not.
> The term can also be used by people who consider it unethical to even give programmers the possibility to hide nefarious code from users, regardless whether they actually do or not.
But that's not what's happening here. They are categorically demonizing innocent people.
> They are categorically demonizing innocent people.
I'm sorry; that was not my intention.
Perhaps I can make a comparison to clarify? As an example, some people think that guns are unethical because they may be seen as an unjust instrument of violence. Even if a particular gun hasn't killed anyone (yet), or even if most guns happened not to be used to kill, then surely it can still be legitimate for people to object to the passive presence of guns, because it gives gun owners the power to kill, and that power may be considered unjust by principle.
Similarly, some people think that non-free software is unethical because it gives programmers the power to do bad stuff, regardless of whether some particular non-free program is actually malware (yet).
(Sorry in advance if I've derailed this discussion into a more controversial subject.)
(continuing the discussion for clarity... no emotional connotation is intended)
I realize this is a comparison for clarification, but isn't it still just 'categorically demonizing innocent people'?
You picked a more controversial topic where more people are likely to agree with the demonization I suppose, but your assertion that the power to kill "may be considered unjust by principle" is not well supported by vague assertions that "some think" guns "may be seen as" unjust instruments of violence. I fully support everyone's right to object to something they see as dangerous; opinion does not constitute principle, however.
To me, something is just or unjust based on whether or not it aligns with or infringes anyone's rights. So, I suppose I might actually agree that a power could be "unjust in principle" if it could be shown that the power could not be used justly - that is, without infringing on anyone else's rights. For some powers, mostly political ones, this is the case. In this case I think guns may be a poor comparison, because they actually can be used in ways which are just (defense, etc.), even if you believe that those cases are unlikely and so desire strict gun control, etc.
In contrast, it may be that producing nonfree software is always 'wrong' (in that it infringes on the supposed rights of the users to understand and modify the program they are running) and therefore having or providing the power to do so would be 'unjust in principle'. If the concern is merely that some may produce malware, and there are actually legitimate reasons for producing nonfree software, then it is not unjust in principle to do so, or to provide someone with said power.
I hope I've understood all that correctly, and restated it well. I'm not sure that I agree with the idea that nonfree software is always bad, but I am open to it. Perhaps what I'm missing is a clear understanding of the specific rights that nonfree software violates.
No worries, I know you (as well as the authors) are simply trying to apply implicit safety measures to counter bad actors. And I'm certainly not offended by you adopting the program. It's my feeling, however, that their approach is horribly wrong and bordering on corruption. I simply don't believe they will have any success when trampling over the good actors in their process of trying to better the world. IMO; If they really wanted to make a dent, they should push for a regulation requiring that browsers provide functionality that enforces a free-software configuration OPTION. Then allow society to decide for themselves (this is a free world after all). I'd even be ok if the default setting was on. But as it sits right now they will get nowhere really fast.
edit: oh and as for the gun analogy... I'm from Canada and fully support gun control (we have it), but I'm not going around and implying that every gun owner is unethical in the process of asking for gun control. That would be shooting myself in the foot!
> IMO; If they really wanted to make a dent, they should push for a regulation requiring that browsers provide functionality that enforces a free-software configuration OPTION.
Sounds interesting. Apart from the regulation part, it sounds a bit like LibreJS[0].
Actually, I got the notion from Stallman's original post 'The Javascript Trap' [1].
"Finally, we need to change free browsers to detect and block nontrivial nonfree JavaScript in web pages. The program LibreJS detects nonfree, nontrivial JavaScript in pages you visit, and blocks it. LibreJS is included in IceCat, and available as an add-on for Firefox."
However I am opposed to that call for action given it's an all-or-none implementation. I feel it's the role of each country to regulate, which is why I expressly suggested it as a configuration option (ideally it could be enforced at the browser level country by country and if not then user by user).
It seems like the thesis here is that whether or not "non-trivial" Javascript (which is just about all Javascript in the wild) should be trusted depends on the presence of an explicit GPL license. If so, that doesn't seem like a reliable heuristic for a script blocker to me.
I'm pretty sure it would be similar to ad-blockers. The initial implementations are trivial and easily circumvented, but as they evolve they become more useful overall.
Plus note that I was just suggesting that it would be more effective than a social movement with 'ethical repositories'. Just imagine if the ad-blocker devs tried the same strategy...
