Simply put, state compression is a more subtle way to reduce the CURD of various states during the operation of an algorithm, thus reducing both time and space complexity.

Let’s use a problem from leetcode to demonstrate the use of state compression in dynamic programming:

This is the stem of the question:

You are given two sets of points, where the first set has size1 points and the second set has size2 points, and size1 >= size2.

The cost of connection between any two points is given by a matrix of size1 x size2, where cost[i][j] is the cost of connection between point i in the first group and point j in the second group. Two groups of points are said to be connected if each point in both groups is connected to one or more points in the other group. In other words, each point in the first group must be connected to at least one point in the second group, and each point in the second group must be connected to at least one point in the first group.

Returns the minimum cost required to connect the two sets of points.

This topic is also a dichotomous graph problem, but it is not the same as匈牙利算法The problem solved is different in that the pairing here is not one-to-one, but many-to-many, as long as each point is connected to a point in another graph

Let’s look at the solution to this problem

Let the first set of points be $size_1$ and the second set of points be $size_2$. Depending on the range of the data, we can use the binary number s to represent a point set, where the kth bit of s is 1 to indicate that the kth point is in point set s, and the kth bit of s is 0 to indicate that the kth point is not in point set s. Use dp[i][s] to denote the first iii points of the first set (the first i points are the 0th,1st,2nd,…,i-1st points). ,i-1 points) with the minimum connectivity cost of the second set of points s (since size1 ≥ size2, the second set is taken as the set of points) in four cases:

• i=0 and s=0:
  Both sets of points are empty, so the minimum connectivity cost is dp[0][0] = 0.

• i=0 and s≠0
  The points in the first group are empty and the points in the second group are not empty, so they cannot be connected, such that dp[0][s] = ∞.

• i≠0 and s=0:
  The points in the first group are not empty, and the points in the second group are empty, so they cannot be connected, such that dp[i][0] = ∞.

• i≠0 and s≠0
  Consider the first set of i-1 points connected to the kth point of the second set of points s. Using $s_{-k}$ to denote the remaining set of points after the kth point is removed from the point set s, then the connectivity cost c has three cases:

  • the kth point of the second set of points s is no longer connected to other points, then c = dp[i][s-k] + cost[i-1][k];

  • the first group of points i-1 is no longer connected to other points, then c = dp[i-1][s] + cost[i-1][k];

  • The first set of points i-1 and the kth point of the second set of points s are no longer connected to other points, then c=dp[i-1][s-k]+cost[i-1][k].

Enumerate the first set of points i-1 connected to any point k ∈ s in the second set of points s. Then the state transfer equation is as follows:

dp[i][s]=min⁡k∈s{min⁡{dp[i][s−k],dp[i−1][s],dp[i−1][s−k]}+cost[i−1][k]}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

var connectTwoGroups = function(cost) {
    const size1 = cost.length;
    const size2 = cost[0].length;
    const m = 1 << size2;
    const dp = Array.from(Array(size1 + 1), () => new Array(m).fill(Number.MAX_SAFE_INTEGER / 2));

    dp[0][0] = 0;

    for (let i = 1; i <= size1; i++) {
        for (let s = 0; s < m; s++) {
        for (let k = 0; k < size2; k++) {
            if ((s & (1 << k))= 0) {
            continue;
            }
            dp[i][s] = Math.min(dp[i][s], dp[i][s ^ (1 << k)] + cost[i - 1][k]);
            dp[i][s] = Math.min(dp[i][s], dp[i - 1][s] + cost[i - 1][k]);
            dp[i][s] = Math.min(dp[i][s], dp[i - 1][s ^ (1 << k)] + cost[i - 1][k]);
        }
        }
    }

    return dp[size1][m - 1];
};

Compression on data structure

The dynamic programming in this algorithm needs no further explanation, in fact, it also contains state compression, for example, compressing a boolean array into a binary number is a common practice, which may not see much improvement in terms of space, but in this problem, we need to represent the state with a boolean array and use this state as the key, then we can only use map to store the key-value pairs, but if we use binary numbers as key, we can use arrays to store key-value pairs.

Eliminate storage of useless intermediate states

Another type of state compression refers to identifying useless states and eliminating them

The calculation of dp[i][∗] for the transfer equation is only related to dp[i-1][∗] and dp[i][∗], so we can save space by using only one-dimensional arrays to keep it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

var connectTwoGroups = function(cost) {
    const size1 = cost.length;
    const size2 = cost[0].length;
    const m = 1 << size2;
    const dp1 = new Array(m).fill(Number.MAX_SAFE_INTEGER / 2);
    const dp2 = new Array(m);
    
    dp1[0] = 0;

    for (let i = 1; i <= size1; i++) {
        for (let s = 0; s < m; s++) {
        dp2[s] = Number.MAX_SAFE_INTEGER / 2;
        for (let k = 0; k < size2; k++) {
            if ((s & (1 << k))= 0) {
            continue;
            }
            dp2[s] = Math.min(dp2[s], dp2[s ^ (1 << k)] + cost[i - 1][k]);
            dp2[s] = Math.min(dp2[s], dp1[s] + cost[i - 1][k]);
            dp2[s] = Math.min(dp2[s], dp1[s ^ (1 << k)] + cost[i - 1][k]);
        }
        }
        dp1.splice(0, m, ...dp2);
    }

    return dp1[m - 1];
};

Come from work and go to work. Never create a mechanism for the sake of creating a mechanism; every mechanism must have practical value.

Since mechanisms are scenario-specific, when the scenario changes, the mechanism has to be upgraded as well. Revisiting and learning the mechanism means extra overhead, so each mechanism has a cost to maintain, and if it is not upgraded and updated as the scenario changes, it will become a meaningless mechanism, and over time it will become common: whatever mechanism is available, but the execution is not good

Stage

• Before the event

  The most important question for managers to answer is what needs to be done, which one to do first and which one to do later, that is, sorting out the priority of tasks, which corresponds to “prioritization” in management

• In the Matter

  Ensuring that things move forward according to plan, which translates into “effective execution” or “project management” in the narrow sense

• After the fact

  To consciously summarize the lessons learned and form some “rules” and “practices” so that similar work can be done better and more smoothly in the future, corresponding to the management of the "process mechanism "

element (e.g. in array, in programming, in programming)

• slight or important, urgent or non-urgent

  For each specific team, there is a limit to the amount of work that can be done at the moment, and simply increasing concurrency does not make the team’s output more efficient

  • Prioritized decision-making steps

    • Critical Emergency Quadrant

    • How to determine if a job is important and urgent

      If you do, does it benefit a lot, the greater the benefit the more important
      If not, is it a great loss, the greater the loss, the more urgent

    • Common scenarios

      Divide daily work into two cases: one is planned and one is unplanned
      For the planned work, you need to focus on how much value and benefit it has in a planning cycle, and schedule in tasks that are of sufficient value to continuously advance
      For unplanned work, since it is an unexpected situation, the first step is to judge the interruption i.e. there are arrangements to immediately follow up the new situation. Interrupting an existing arrangement will definitely affect the work at hand, that is, it will affect what would have been gained.

    • Steps

      First of all, judge the priority of the work to specify their own plans. For the work within the plan, see if the benefits are large enough, the greater the benefits the more important, the greater the benefits, the greater the need to give matching priorities, resources and attention; relatively small benefits, put into the todo list, as a to-do task
      For unplanned work, see if the loss is large enough, the larger the loss, the more urgent it is. If the loss is large enough, the task is scheduled as urgent, with “stop loss” as the core purpose. If the loss is manageable, put it in the list of planned work, and use the gain as the measurement indicator.

  • Decision Points

    • Goals need to be consistent

      As mentioned earlier, judging whether a task is important by looking at the benefits, what frame of reference should be used to measure the benefits?
      The answer is goals, which contain our most important demands and expectations over a period of time, and are the axes of measurement that affect the size of our work gains. The clearer the goals, the stronger our sense of direction at critical moments.

    • The evaluation latitude of the task is flexible

      For a task, the three elements of progress, quality and effectiveness can be mutually exclusive, so when disassembling a task, different expectations of progress, different requirements for quality and different expectations for effectiveness can lead to changes in expected time and priority

    • Communication is indispensable

      Superiors are more likely to tell you what they want than to offer to tell you what they are willing to trade, not exactly because they are “profit-oriented,” but because it is not their job to assess the impact and offer the appropriate solution, which is something you know and can do.
      Superiors can not judge how big the impact on your i.e. arrangement, you need to give an assessment, if you do not say anything, the default is no impact.

• Process Management

  • How to ensure effective project execution

    • Sufficient Conditions Perspective

      Just list all the points for effective execution and follow them, but it’s not realistic

    • Necessity Perspective

      Although we cannot give all the conditions for effective project implementation, we can certainly put forward some points in long-term project management, and as long as these points are not achieved, the project will be difficult to be implemented effectively

  • Four Barriers to Effective Execution

    • unclear objectives

      • Unclear and unmeasurable goals
      • Objectives are not aligned and have deviations
      • The target has changed and there are no synchronized related people

    • Absence of General Manager

      • Three questions

        • Is there a clear leader for each team involved in the project
        • The person in charge and all project team members, are they clear about the individual aspects of the person in charge
        • Whether the project has a sole general manager and whether the general manager is effective

      • Two easy areas of ambiguity

        • Inconsistent understanding of “responsibility” among principals
        • Total person in charge is invalid or missing

    • Lack of mechanism for effective operation

      • Question

        • Some managers have seen some good people’s excellent performance, too superstitious people’s initiative and career level, once the problem, always feel that people can not. In fact, the ability level of team members is normally distributed, even if it is true that people can not, from not to line also need the process, at this time is the time when managers should do something
        • When people don’t work, they like to use processes and mechanisms. As a result, managers develop a full set of processes for their teams to follow, and because of the high cost of learning and implementation, it is painful for employees to follow, thus causing the processes and mechanisms to sleep

      • Summary

        • Too much reliance on human initiative, lack of process and mechanism
        • Although there is a mechanism, there is no one to monitor the implementation
        • Employees are reluctant to perform even though there is supervision of the implementation

    • Inadequate communication

      • Lack of proactive awareness and communication
      • Insufficient awareness of notification and failure to notify all relevant personnel
      • Insufficient awareness of the closed loop, broadcast out, the other side received by default

  • The four elements of effective execution

    • Clarity of purpose
    • Clear responsibility
    • Sound mechanism
    • Communication in place

• Process Mechanism

  On the one hand, many managers hope that a lot of management work can be guaranteed through processes and mechanisms to improve their management efficiency; on the other hand, they are at a loss because a large number of processes and mechanisms are lying in the document and sleeping.

  • Steps of process mechanism establishment

    As managers, trying to get employees to share the work we have at hand either by laddering or by process mechanisms

    • The so-called rely on the gradient, that is, the team has a very high degree of competence within the people who can help us to deal with this matter
    • The so-called relying mechanism is to design a set of programs to specifically respond to the problems that arise in a certain scenario, and use this set of programs to guide employees to do this kind of work
    • If the original intention is talent development, then it is reasonable to go and bring employees along to develop the ladder. But if the original intention is to reduce the burden of managers, that instead should rely on the mechanism. And, training echelon is always far water, rely on the mechanism to solve the problem at hand is near thirst, how good the echelon can not mention the role of the mechanism

  • The process of establishing the authorization mechanism

    • Clearly defined objectives

      Clarify what problem in what scenario the change mechanism is intended to solve.
      One of the major characteristics of the mechanisms is that the scenario-based characteristics are very obvious, because they are generated to deal with specific problems in specific scenarios, such as the service alarm mechanism

    • Refine the key points to deal with the scenario

      Distill the key aspects of responding to that scenario from you and experienced people, so the distillation of these key points will be much easier if there are successful experiences.
      It is not recommended to organize a detailed operational documentation, and a step-by-step complete documentation is more important than the key point refinement, which will make the implementation costs lower, but also more operational.

    • Identify the supervisor

      That is, who will ensure that the mechanisms are implemented, and at what point will the key points be checked, and how well each process and mechanism is implemented, and who will check and confirm

    • Evaluate operating costs

      Confirm that the mechanism is operational for the implementer, that it was created to simplify the work, and ideally to achieve “autopilot” for management, and that if the creation of the mechanism results in higher operational costs for the implementer, it is necessary to reflect on the rationality of the mechanism

    • Consensus Building

      Communicate and get consensus with relevant executors

  • Principles of process mechanism establishment

    • Operability

      That is, the principle of simplicity, that is, the mechanism should be based on the principle of the lowest learning and operating costs, which is the most important principle, if the mechanism established does not have operability, then even if the managers feel perfect in themselves, the more problems they can cope with and solve, they should also be decisively abandoned

    • Hit only the key points

      To establish the process mechanism, it is not necessary to describe all the details in full, no one likes long words, especially technical people, you just need to tell everyone what to do in those few key points

    • Clearly defined to the person

      Principle of accountability. The question of who will follow up on each key point should be clearly agreed upon, and cannot rely entirely on human consciousness

    • Practical principles, to upgrade with the scene change

      Come from work and go to work. Never create a mechanism for the sake of creating a mechanism; every mechanism must have practical value.
      Since mechanisms are scenario-specific, when the scenario changes, the mechanism has to be upgraded as well. Revisiting and learning the mechanism means extra overhead, so each mechanism has a cost to maintain, and if it is not upgraded and updated as the scenario changes, it will become a meaningless mechanism, and over time it will become common: whatever mechanism is available, but the execution is not good

  • Frequently Asked Questions about Process Mechanisms

    • Is the more mechanisms the better

      The mechanism is not more the better, but the less the better. This and the mechanism of the principle of simplicity, practical principle of the same line. The establishment of the mechanism does not solve the problem, the implementation of the mechanism to solve the problem, and the establishment of the mechanism, the implementation and post-maintenance are costly, so do not greedy more

    • People are reliable or mechanism is reliable

      Human performance is more volatile, the point of the mechanism is to go to the lower limit of ensuring things when people are not reliable

    • The mechanism is reliable enough, is it not necessary to motivate

      Mechanisms are a lower limit to ensure things are done, and are not a substitute for incentives

• How to execute efficiently

  A company owner once said that a competitor’s team was 996 every day, so he wanted his team to be 996 as well.
  The question is, what does he hope to accomplish with 996.
  He actually did not know why he wanted 996, just wanted everyone to work hard, as for what purpose to achieve, he was not clear.
  Remember, 996 is only a means, and the means must serve an end, and when the end is not clear, the effectiveness of the means can not be judged. We often rely on the so-called “taken for granted” to make decisions, often fail to achieve the desired effect, and even contrary to expectations, the only effect is the stage of comfort

  • Efficient execution = individual capacity * cooperative effectiveness * directional effectiveness * working hours

    Many managers have put the focus on the length of work, which is a general incompetence, tired of the three armies

    • Individual capacity = strength * willingness = capacity development * employee motivation

    • Cooperation effectiveness

      Through the division of labor so that everyone can work together in one place, while relying on the trust and tacit understanding accumulated in general to make the team have a good level of collaboration

    • Directional validity

      • The problem of talking about implementation in isolation from the goal

        • Incentive failure

          • Work repeatedly switched, the previous discussion evaluation design development have become a sunk cost
          • Time is getting tighter and the challenge is getting bigger
          • Employees think management is not thinking clearly and question management’s ability

        • Collaboration disorders

          • Clear and cognitively consistent goals help maintain a unified work pace
          • Lack of clarity in goals can lead to inefficient communication

        • Busy and ineffective

          • Clearly defined objectives
          • Define the metrics of the goal

- Echelon planning- Echelon training- Selection and selection of training objects- PrincipleMaintain consistency in talent selection and team building- Ability: Whether the individual's ability matches the business characteristics, and whether its ability potential can be cultivated- Collaboration: their awareness and ability to collaborate, and whether they match the team's requirements and expectations- Culture: Its behavioral style and values, and whether they match the team's cultural values- Value people who complement each other  People with similar styles collaborate more effectively, but lack richer insights and perspectives.  - Cultivate the people you find- Align expectations  Managers and employees agree on the expectations of the employee's personal development. A common way is IDP, or personal development plan.  - No commitment principle: that is, do not promise any benefits to employees  On the one hand, being able to become the core backbone of the team depends on one's influence, not commitment. Second, if you can't deliver, this person may be lost.    PS: You don't promise any benefits, why should he listen to you? This person may be lost now  - do a good job of authorization  Ability and influence are accumulated in actual combat, which requires training objects to provide space and opportunities to do things.  - In advance- Examine the original intention  Be clear about what you want to gain in this authorization, whether you want to do something or bring people out  - Clear expectations  Let the training target clarify the expectations of managers  - Listen to his train of thought  After giving a good account of the authorized task, listen to his views and ideas on this matter first, and you can generally judge that the reason why he is independently responsible for this work is not reliable, and it is also a good recipe for risk control  - Important agreement  Managers should make an agreement on the things they are particularly concerned about and the training objects, such as how to check whether it is completed and when to check  - In progress- Check the progress- Give support- After the fact- evaluation results  For any authorization, give reasonable and well-founded evaluation and timely feedback on the work results and performance of the authorized object  - Insight advantage  What are the outstanding advantages of authorized objects in the entire authorization process?  - Positive feedback  Give some positive feedback on the work of the authorized object  - An improvement  Give one or two suggestions for improvement  - Build feedback- Establish a periodic communication mechanism to make communication routine- Review IDP, evaluation and feedback on the implementation of IDP, in order to reflect the greater value of IDP- Arrange a second mentor

• Culture

  What is team culture?

  There is no single clear definition of team culture itself, and every team has some conventional ways of working and judging right and wrong, so that in a team, even if no one tells you what is right and wrong, you are in general It can be clear what to do and what not to do.

  The value of team culture

  A clear team culture will bring at least four values to the team: efficiency, space, belonging and endurance  - Efficiency is the effect brought about by the orderliness of culture    Because the team culture contains conventional work standards and decision-making basis, and team members have consensus on this, there is no need to consult the superior for everything. A team that emphasizes "safety", everyone will consider safety measures at work, and a unified code of conduct and tacit understanding of collaboration will greatly improve efficiency      - Space is the result of cultural orientation    In line with the premise of Value Proposition, employees can choose their own work methods, and even work content, which is a lot of initiative by the staff to provide independent play space      - Belonging is the effect of cultural screening    Team culture acts as a filter. People who agree with the culture will continue to join, while those who do not agree will gradually fade out      - Endurance is the result of cultural continuity    Culture is a relatively stable element for an organization. It can be passed on between new and old employees and will not change due to the changes of individual personnel

  • Steps to build team culture

    • Refining

      Put it in the right words

      • Misunderstanding

        The culture of your team has less to do with what culture you like and more to do with who you are

          Facing problems and always complaining, then emphasizing a positive culture is not feasible  - In the face of cooperation, there is always confrontation, so emphasizing close cooperation is not feasible  - In the face of work, always wait passively, so it is not possible to advocate initiative  - In the face of subordinates, always indifferent, then emphasizing warmth is not enough

        • The more concise the better

          • The purpose is for everyone to understand and remember, not the fewer words the better

      • Way

        • Extract from your own excellent qualities, thereby amplifying the excellent qualities to the entire team. The qualities you do not have are not suitable for cultivation as a team culture

    • Advocate

      Disseminate the refined culture to the entire team, including superiors and brother teams

      • Not to talk to anyone, but to be aware of the team culture in some public or private occasions

    • Practice

      • Reasons why employees don’t buy into the culture

        • When culture is refined, it is not extracted from the manager himself. The manager cannot lead by example, and his words and deeds are inconsistent
        • Did not combine team culture with management work, just talk without practice, did not let employees feel the benefits of practicing culture

In this blog, we will briefly talk about the application of maps, and then transition from displacement maps to the expression of aggregates.

Application of Geometry

In GPU programming, the map is equal to the memory + range query in our CPU programming. We can use ByteGraph to paste the data we need in the calculation process and perform range queries on the data.

Therefore, the texture can not only be used to store the color information of the point on the object itself, but also can store information such as environment information, normal information, displacement information, etc.

Environmental map

As shown in the figure below, the reflection of the environment on this teapot needs to be calculated in real time, but the environment is infinity, so the relative position of the teapot to the environment remains unchanged, and the reflection result is also unchanged, so we can calculate it in advance. Then save it in the map. When rendering, just mix the color map of the teapot itself and the sampling result of the environment map.

Bump map (normal map)

If our geometry itself is relatively smooth, we want to create a sense of unevenness on a smooth surface, such as creating a sense of asphalt road on a plane, we can use the way of bump mapping, because the so-called bump feeling we see is also caused by the different angles of light reflection. We can directly intervene through bump mapping when calculating the reflection angle.

Displacement mapping

Just now we said that we can change the reflection angle of each point on the smooth surface by means of bump mapping to create a bump feeling. Then we can actually change the position of the point directly through displacement mapping when rendering. Rendering effect.

The difference between this method and bump mapping is that bump mapping does not change the position of the point, but only changes the direction of the normal, while the only mapping is to directly change the position of the point, which stores the position of the corresponding point based on the original position.

So here is a question involved, I changed the position, how to calculate the new normal?

The displacement map here is actually a representation of geometry, an explicit representation

Representation of geometry

There are many different ways of representing geometry, but there are mainly two, one is explicit and the other is implicit.

Implicit representation

The so-called implicit way, which represents the relationship satisfied by the points on the set, theoretically we can get this geometry by finding all the points that satisfy the relationship.

Common implicit representations are:

• algebraic surface
• level sets
• distance functions

algebraic

For the implicit representation, it represents the relationship satisfied by the points on the set, so it can be very simple to determine whether a point is on the surface of the geometry, and its disadvantage is that it is difficult for you to see what it represents, as shown below

** Here we may have a question, that is, can this way of doing things only represent relatively simple and regular graphics? **

In fact, implicit way can represent very complex graphics, here to provide two ideas:

The first idea is that if we have studied advanced mathematics, any function can be Fourier expanded in the way that multiple functions are added together. Similarly, we can add enough functions to fit any function, as long as you have this knowledge reserve and willing to spend time.

The second idea is more like the idea we have when we make charts, using multiple simple graph splicing or elimination methods (that is, Boolean operations) to do it, as shown in the following figure:

distance

As shown in the figure, this method is a variant of the boolean operation of the second idea we just mentioned

Signed Distance Function (SDF) is a mathematical representation commonly used in Computer Graphics and geometric processing. It is used to describe the shortest distance from a point to the surface of a shape, and also contains the position information of the point relative to the surface of the shape. The value of SDF can be positive, negative or zero:

1. When the point is located on the surface of the shape, the value of SDF is zero.
2. When the point is located inside the shape, the value of SDF is negative, indicating the shortest distance from the point to the surface of the shape.
3. When the point is located outside the shape, the value of SDF is positive, indicating the shortest distance from the point to the surface of the shape.

SDF is widely used in many applications, such as ray tracing, collision detection, fluid simulation, etc. Its advantage is that it can efficiently calculate the distance between points and shapes, while retaining the position information of points relative to shapes.

In the above figure, in the process of merging two balls, we first calculate the distance function of any point in space relative to the surfaces of the two spheres, and then weigh it, leaving the point with 0 as the result

If the three-dimensional one is more difficult to understand, we can use a two-dimensional diagram to see:

level

This method is also a variant. For the case where we cannot write an SDF, but can only write a rough table map, how do we get the result after fusion? Take a line as an example:

Explicit representation

The explicit representation is to directly give information to the out-point surface, or through parameter mapping, such as our very common Mesh. The displacement map we just mentioned is actually a mapping method, so we just said it is an explicit geometry representation.

Common explicit representations are:

• point cloud: point cloud
• polygon mesh: we common mesh
• Subdivision, simplification, regularization: subdivision, simplification, regularization

The disadvantage of explicit representation is that it is more difficult to determine whether a point is on the surface of the geometry. As shown in the figure below:

Point cloud

This is easier to understand, is a large number of points, these points do not need to connect, as long as enough, you can not see the gap.

Because it is a series of points, it is good to store it in an array, but the amount of data is very large

Polygon mesh

Specific usage scenarios

Bezier curve

A Bezier curve is a method of determining a curve from several points. It has very common applications. For example, if we use xmind, when we establish a connection between different labels, it will give us four points, of which two The fixed point is the position of the two labels that need to be connected, and the other two points are given to us to drag, and after dragging, we can change the shape of the curve. This curve is a cubic Bezier curve.

Bézier curve can have n times (n is greater than or equal to 2), we take quadratic Bézier curve as an example to explain, and so on

As shown in the figure, suppose we have three points. Our three points determine that our curve starts at $b_0 $and ends at $b_1 $. The tangent direction of the starting point needs to be $\ vec {b_0b_1} $, and the tangent direction of the end point needs to be $\ vec {b_1b_2} $

Suppose we have a point that moves from $b_0 $to $b_1 $in time, then at any point t, its position should be:

The ratio of $b_0b_0 ^ 1 $to $b_0 ^ 1b_1 $, the ratio of $b_1b_1 ^ 1 $to $b_1 ^ 1b_2 $, the ratio of $b_0 ^ 1b_0 ^ 2 $to $b_0 ^ 2b_1 ^ 1 $are all t/(1-t)

Quadratic Bézier curve is actually to use three points to get two new points proportionally, and then to get a point proportionally on the connection of the new two points

The cubic Bézier curve is actually four points that determine the curve, and the way it connects gradually becomes one point from four points

The higher Bezier curve is actually the recursion of the above process:

We just talked about how the Bézier curve is obtained, so how to express this process in a formula? The following is the result of the quadratic Bézier curve

The result is that we actually take the coefficients on the right side of the equation and it’s the expansion of $ (1 - t + t) ^ 2 $

For a higher-order Bézier curve, the formula is as follows, where B represents the expansion of the binomial distribution, that is, the probability of getting a result each time is t, then the probability of j results in n independent replicates:

Joining of Bézier curves

When our curve is a very high-order Bézier curve, it is actually difficult to control, as shown in the figure below:

So we can take the method of splicing multiple low-order Bézier curves:

Bessel surface

A Bézier surface is a surface controlled by 4 * 4 = 16 points:

As shown in the figure below, first of all, we can get a Bézier curve from the four points of each line horizontally. At this time, we get four Bézier curves, and then we imagine that there is a plane cut vertically, and four Bézier curves get four points, these four points can then determine a Bézier curve, and we can get a Bézier surface in the process of moving this numerical surface:

This process is actually a recursion process.

Mesh

The so-called mesh subdivision refers to splitting the polygons in the mesh to obtain a smoother effect

From the definition, we can see that this mesh subdivision actually has two things to do. The first thing is to split the polygon, and the second part is to adjust the split polygon, otherwise the simple split will not be The smoother effect will be obtained, for example, we connect the midpoints of the three sides of a triangle to obtain four triangles, but these four triangles are still on one surface, and there will be no smoother effect

Loop

It’s called Loop only because the person who invented the algorithm is called Loop.

The first step of the algorithm is to connect the vertices of the three sides of the triangle, and we have four triangles.

Next we need to update the position of the dots to make the split face smoother:

For the three new points, we update the coordinates in the following way

For the original three points, we update as follows:

Catmull-Clark

The limitation of Loop subdivision is obvious, it can only split triangles, so how do we split polygons?

It can be seen that only the first split will increase the number of singularities, that is, only the first time the triangle is split into three quadrilaterals, how to split the quadrilateral is a quadrilateral.

So how do we use the updated position of the new quadrilateral?

Mesh

Edge collapse

In fact, one edge is deleted, or the two vertices of the edge are merged into one.

、

This scheme sounds simple, but there are two problems, which side to remove, and where the position of the merged point can introduce the smallest error.

Note that in graphics, many of the so-called most only mean acceptable effects

Here we introduce a new concept called quadratic metric error, that is, the distance from the plane L2 where the distance is deleted and the minimum L2 distance

We only need to calculate the quadratic measurement error of all faces, and then select the smallest deletion in turn.

There is actually a problem here, that is, after each deletion, it will affect the position of some points, thus affecting their secondary measurement error. We need to recalculate and reorder to get the smallest one for deletion. Here we need to use the priority queue data structure, get the team head for deletion every time, then update the affected secondary measurement error, and then get it from the team head.

Before we said, each point has its own UV coordinates, and then find the corresponding point on the map through the UV coordinates, and then take out the value. We can interpolate the pixels in the middle of the triangle through the center coordinates.

But the problem is, if our map is too small, it may cause the model to have multiple points corresponding to the same UV, which will cause blurring. On the contrary, if the map is too large and the UV coordinate difference is too large, it will cause the final jagged or moiré effect.

This article will talk about how game engines solve these two problems.

Basic sampling rules

(X, y) are the pixel coordinates on the screen, (u, v) are the texture coordinates corresponding to the points on the model

1
2
3
4

for each rasterized screen sample (x,y): 
    (u,v) = evaluate texture coordinate at (x,y) 
    texcolor = texture.sample(u,v);
    set sample’s color to texcolor;

What happens if the sticker is too small

Solution 1: Texture enlargement

This scheme is the simplest scheme, the texture is too small to be enlarged by a certain algorithm.

Here we introduce a new concept called texel, which is a pixel on a texture, and our UV coordinates take a texel from the texture

There are three common algorithms for texture amplification: Nearest, Biliner, Bicubic

Let’s take Biliner as an example to explain the process of texture enlargement.

Suppose we want to sample the value of the red point in the following figure, we select four points around the point, then calculate the distance between the point we want to sample and the center of the four texels, and interpolate with this distance:

What happens if the texture is too large

First, let’s look at a rendering:

In the near place, because the texture is too large, some details are lost, resulting in jagged teeth, while in the distance, molar patterns are produced due to too many details. (The reason belongs to personal understanding, but the effect is right)

When the texture is too large, it actually has the following effect:

That is, when we use the “basic sampling rule” mentioned at the beginning to find a certain textured element on the map, we find not one, but a piece.

In other words, if the texture is too small, multiple pixels will find the same texel, and if the texture is too large, one pixel can match multiple texels

Option 1: Supersampling

The algorithm of this oversampling technique is actually the same as the texture amplification just now, but for the case of too small a texture, oversampling is aimed at the original map, which is used to generate a higher resolution original map, and when the map is too large, A pixel may correspond to multiple texels, so the sampling result of the pixel from the map is jointly determined by multiple texels around it.

The following picture shows the effect of super sampling:

It can be seen that supersampling is indeed effective, but the problem is that the calculation cost is too high, and each pixel needs to calculate the value of multiple points and interpolate.

Option 2: Find the average

Supersampling calculation is troublesome, we can simply average it over a range

Option 3: Mipmap

This is a faster range query, is a kind of space for time idea, we need to calculate the results of the range query at different sampling frequencies in advance and store them in the map, that is, to generate maps with different degrees of low resolution. When the map is too large, go directly to the smaller resolution map to find it.

Mipmap is a texture mapping technique used to render textures with different details at different distances and angles. A Mipmap texture is composed of multiple texture images of different resolutions, each image is 1/4 the size of the previous image. The level of Mipmap refers to the first few images in the Mipmap texture, counting from 0, and 0 is the original texture image.

This plan has two key points:

1. Generate different levels of Mipmap
2. How to determine which level of Mipmap to use

The first point is the image compression algorithm, there are many, we do not go into detail, nor rendering specific knowledge, we mainly look at how to determine the level, that is, the “D”

The level of Mipmap can be calculated using the following formula:

level = log2(max(w, h))

Where w and h are the width and height of the texture image, and max (w, h) represents the maximum value in width and height. This formula means that each Mipmap level image is 1/4 the size of the previous level image, so use the logarithm function to take 2 as the base when calculating, and you can get the size of the largest level.

For example, if the size of the original texture image is 512x512 pixels, the Mipmap texture contains 8 levels, which are level 0 to 7. Level 0 is the original image, level 1 is a 256x256 pixel image, level 2 is a 128x128 pixel image, and so on.

We need to use a reasonable Level to have a good price comparison effect. The higher the level, the less details. If we use the oversampling method with too little texture, it will be more blurry:

Plan 4: Anisotropic

Mipmap actually has limitations, it can only do mipmap on textures in the positive direction.

Because of his limitation, on the one hand, when sampling, many unused points will be used, increasing the amount of calculation, and secondly, some pixels correspond to rectangles covering multiple texels, resulting in oversampling blur.

For the case of rectangles, we can use Anisotropic Filtering

Scaled square maps are on the diagonal, other locations store some width compressed and length compressed maps

• Good performance

  • Direction: Manage the planning phase, define what constitutes good performance, and plan the team to match it
  • Lead people: that is, the team building stage, continuously improve the team’s ability to achieve performance, including individual ability and team ability
  • Work: task management stage, how to implement the plan through good control and produce results

• High efficiency

  • Look at the direction: the clearer the goal, the higher the efficiency
  • Lead people: The stronger the individual ability of team members, the higher the level of cooperation among team members, and the higher the team efficiency
  • Work: The more efficient the processes and rules, the more skilled the operation, the higher the efficiency of work

• Sustainable

  • Look at the direction: the rationality and effectiveness of team planning
  • Lead people: perfect layers and mature team culture
  • Work: The tenacity of the team is honed, and honed cannot be separated from continuous work

Team building elements

Capacity

• Components of capacity

  • Capability three core

    • Knowledge

      Knowledge and understanding of information, generally measured in breadth and depth

    • Skills

      The skill and ability to perform and perform a job, generally measured by proficiency

    • Talent

      • Spontaneous high-frequency patterns of thought, action, and feeling, including character traits, etc

  • Three dimensions of work ability

    • Professional competence

      • For technical people, it generally refers to technical ability, which is different for different teams. Can be measured in breadth and depth

        Breadth: Represents how large-scale things can be done
        Depth: Represents how complex things can be done

    • General ability

      Such as: structured thinking ability. Teamwork ability. Project management ability. Quick learning ability

    • Personality strength

      Personality strength usually refers to a person’s stable attitude and performance in the face of a certain situation, such as facing difficulties, perseverance, and positivity.
      These personality forces are sometimes crucial to the ability to get something done, but they are not cultivated overnight. They often come from everyone’s long-term life and work experience, and once formed, they are difficult to change.
      But hard to change doesn’t mean it can’t be changed. In order for team members to tend to a certain common temperament, managers need to unequivocally advocate the character that the team promotes.

• Objectives of capacity building

  It is best to focus on learning goals, rather than scattered knowledge points

  • For example

    • Professional competence perspective

      Prior Art Series
      Core technology thematic learning:
      Architecture topics
      Storage topics
      Front-end topics
      Security topics
      Operation and maintenance topics
      Automated Testing Topics
      Expand new technologies
      New technologies needed for business development
      Technology in the field

    • General ability perspective

      Project management capabilities
      The ability to bring people
      Communication skills

• Standards for capacity building

  What is the original intention of improving employee capabilities? Different original intentions decide what standards to develop, and then write this standard into the employee’s personal development plan and agree with them

  • Avoid unclear expectations leading to high expectations or poor results
  • Reasonably arrange the energy input of managers, and reasonably allocate the energy invested in different training objects

• Methods of capacity building

  7% rely on work practice, 2% rely on mutual communication, and 1% rely on listening to lectures and reading books for self-study

• Employee Willingness to Learn

  How to stimulate the motivation and willingness of employees to learn?

  • Push: give pressure, push him to learn

    Provide clear job requirements

    • Establish learning mechanisms. Such as mandatory compliance with learning rules and completion of learning tasks

  • Pull: Give direction and guide him to learn

    • Set an example
    • Equipped with tutors
    • Develop a skills map for the team to grow

  • Let go: give room to play and learn independently

    • Give employees the opportunity to shoulder heavy responsibilities
      Give employees space to think independently
    • Give him confidence and patience, allow him to make mistakes and take detours

• Two beliefs in ability development

  • Believe in the diversity of employees’ abilities, that is, see the differences and value the richness

    In the era of knowledge economy, the creativity of employees can bring greater value to the team, and creativity often comes from the collision of differences

  • Believe in the system of team capabilities, that is, appreciate differences and value complementarities

• Motivation

  Individual combat effectiveness depends on two elements: individual ability and willingness to use ability

  • The challenge of employee motivation

    Upward communication, employee motivation and team cohesion are the three most challenging management themes in managers minds

    • Incentive cognitive non-system

      Understand some motivation skills and methods, but do not understand the overall picture of motivation, nor the reasons behind it, so there is no systematic incentive plan

    • Lack of incentives

      Real material incentives are not controlled by managers, and the spirit of “painting a pie” motivates employees without buying it, so there are few incentives available

    • Incentives don’t get the point

  Maslow’s Hierarchy of Needs

  Does this theory apply to China? Is there anything that needs to be changed according to local conditions?

  Three Stages of Employee Motivation Development

  • Instinct

    Mainly derived from the desire for survival and security, the hierarchy of needs is at the bottom of the “Maslow Hierarchy of Needs Model”.

  • Outside drive

    The underlying philosophy is “seek reward, avoid punishment” - carrots and sticks - a widely accepted form of motivation in the industrial age for nearly 200 years. Its central purpose is to make employees “obey”.
    At the same time, Taylor’s “scientific management theory” believed that workers were like parts in a machine, they just needed to do the right thing in the right way at the right time to ensure that the machine worked well. In the industrial age of assembly lines, workers were required to do “obedience” rather than ingenuity

    • Existing problems

      • For most managers, material incentives are beyond the control of managers except for the pie

      • Lead to the disappearance of intrinsic motivation and reduce employee self-motivation

      • Causing a decline in performance

      • Kill creativity

      • Suppress good deeds

        For example, when a team’s rewards for excellence are not obvious, but the penalties for mistakes are severe

      • Encouraging fraud, shortcuts, unethical tactics

      • Addictive, making employees addicted to external incentives

      • Breeds short-sighted thinking

      • Will gradually be immune, “Those who are good at using authority will not be easily angry, and those who are good at using grace will not give indiscriminately”

    • Non-material incentives

      Such as praise and positive feedback, the negative harm of external incentives is low, and it is a commonly used external means

      • Specific principle: the content of praise should be specific

      • Disclosure principle: public recognition

        • Give praised employees more incentive
        • Tell everyone in the team what behaviors and values the team agrees with and advocates

      • Timely principle

  • Inside drive

    • Three aspects of internal drive

      • Take control of your own life, i.e. “autonomy”
      • Extend your abilities, i.e. “specialization”
      • Make life more meaningful, i.e. “purpose”

    • Using internal drive to motivate employees is an unavoidable choice

      • Choice of times

        In the era of knowledge economy, the creativity of employees can create more value for the company, and creativity requires autonomy and difference, which is almost the opposite of the concept of the industrial age.

      • People’s choice

        With the development of China’s economy and culture, material rewards and punishments and the evaluation of others have become less important than before. Many professionals have their own unique work values, especially whether they are willing to do it

    • How to excite

      • Increases employee autonomy, that is, gives employees a certain degree of autonomy and control

        • The first is the freedom to work when and where

        • Secondly, the degree of freedom of work content

          Employees can choose their own work content to a certain extent

      • Improve the precision of employees, so that employees continue to grow

        Specialization here does not emphasize setting goals to become an expert, but emphasizes the process of self-involvement, creating conditions for employees to be willing to self-invest

        • Clear work objectives

          The clearer the requirements for the employee, the more willing he is to put in the effort

        • The goal should be slightly challenging

          Demanding too much from employees can lead to anxiety, and too little can lead to boredom

        • To be able to use its advantages

          Everyone is willing to do what they are good at, and if a job can give full play to the unique advantages of employees, employees will definitely be enthusiastic.
          Advantages can have many levels, we may not be able to meet the work content expected by an employee, but we can also consider the behavior mode and thinking mode, such as letting it do some communication

      • Improve the sense of meaning and mission of work

        While not rejecting benefits, pursue maximizing “value” and hope that the work you do is meaningful and valuable
        There is always a group of people who leave because their work is worthless. They are not hypocritical, but really need their work to be valuable

  • Employee happiness

    Total Sustainable Well-being (PERMA)

    Not just emotional happiness, but a flourishing state of life that Martin Seligman calls “blooming.”

    • Positive emotions

      What kind of team atmosphere are you creating? Are team members trusting each other, cooperating happily, helping each other or complaining and accusing each other?

    • Human relationships

      What have you done to improve your employees’ sense of belonging and inclusion? Are there activities and mechanisms involved to make employees more willing to support each other, such as more commonly, have you assigned a designated mentor to each employee?

    • Put in

      Whether the conditions are created for employee autonomy, such as setting clear goals, giving them appropriate challenges, and supporting them to play to their strengths

    • Achievement

      Meeting challenges and achieving success is something that most engineers enjoy, but it requires a premise, which is our characterization of achievement.
      Make daily work projects and milestone large-scale projects

    • Meaning of life

  • Design of incentive schemes

    • Incentives

      Is the incentive plan aimed at an individual, a group or the entire team?

    • Incentive purposes

      Clarify the effect that the incentive plan wants to achieve

    • Incentive type

      Mainly to distinguish whether the plan focuses on short-term effects or long-term effects

    • Incentives

      Choose what motivational elements to work on, each element of the three phases of the drive, all five elements of PERMA can be considered

    • The situation of the incentive object

      To be motivated to have a sufficient understanding of the object, such as the object of the incentive base line is what, only significantly more than the base line has the incentive effect, if below the base line, there will be obvious negative incentive

    • Reasonableness assessment

      • Consider the cost performance ratio from the cost
        Risk: that is, who will create obvious negative incentives

    • Incentive timing

  • Responding to employee motivation challenges

    • Cognition of motivation: upgrade from a single incentive dimension to a three-dimensional incentive system
    • Incentive efforts should be used in peacetime, rather than relying on some temporary stimulation programs
    • Understand the elements to consider when designing an incentive program and be able to operate

Division of labor

• The purpose of the division of labor

  Many people’s first reaction is to improve efficiency. However, if efficiency is pursued, the division of labor should be reduced, because the more people cooperate, the lower the efficiency, so some teams’ performance will increase after layoffs.

  In order to achieve scale, the division of labor is not for efficiency, but for scale

  • Collaboration: Division of labor is the means, collaboration is the end
  • Achieve specialization

• Common organizational structure

  • Matrix structure

    Employees team up according to their roles and are divided into different role teams, each team has its own person in charge. When you want to do a project, there will be a special project manager to apply for manpower from each team leader.

    • Pros: The professionalism of the team in each role is high
    • Disadvantage: The project is relatively inefficient to execute

  • BU type structure

    All personnel and resources for a business are deployed in a unified manner. No matter whether the business department is large or small, it has a full range of roles.

    • Advantages: High cooperation efficiency
    • Disadvantage: Redundant resources

• Common division of labor issues

  • Blurred division of labor

    This is the most common misunderstanding, and even some managers in order to let everyone complement each other, take the initiative to undertake, enhance mutual assistance, but also to blur the boundary, the so-called “de-boundary”.
    However, it should be noted that any blurring of boundaries that is not premised on a clear division of labor will backfire. Because:
    Only a clear division of labor can bring a sense of belonging and be willing to take the initiative to give more. As long as you understand what you should do, you can know what you do is “proactive”

  • Division of labor stability issues

    Only stable division of labor can reflect the value of division of labor

    • The continuous adjustment of the business will cause the division of labor to change constantly
    • Long-term stable division of labor, while bringing expertise, will also bring limitations and laziness, fragmented vision

• Virtual organization

  • Advantages

    • Efficient execution

      The organization of virtual teams is generally for dedicated people, focused goals, and efficient execution

    • Reasonable allocation of resources

      For many tasks, it is not worthwhile to form a dedicated team, but from time to time new projects need to be set up. Virtual teams provide a solution for such needs. Everyone can come together to do things, but not as a team with an independent budget

    • Maintain a sense of belonging

  • Principle

    • Specialized people, not too many

    • Recognize employee value

      Now that you have agreed to your employee’s participation in this virtual organization, you must recognize his value regardless of whether the performance is what you value most.
      Many managers often depress employee performance on the grounds that employee output is not of great value to their team, which is a typical example of letting employees bear the consequences of managers’ decisions

    • Clear goals and responsibilities

Collaboration

The direction of effort is: tacit understanding like a person

How to improve the level of collaboration

- Overall coordination  - Evaluate the level of collaboration and identify work priorities    For the level of collaboration, it is difficult for us to give a measure, but the level of collaboration is excellent and medium, which can be roughly felt.    Mainly to answer a question: what are the main problems in the current collaboration, and what can be done to significantly improve the level of collaboration  - Advocate and promote various tasks    Implement the concept of "collaboration is important" to team members, and promote some measures for employees to switch height to promote collaboration  - Overall coordination, provision of arbitration    Collaboration is a dynamic process. There will definitely be differences and frictions among employees in the process of collaboration. Managers should take resolving differences as an opportunity to improve collaborative relationships.    Resolving differences and contradictions in cooperation is precisely the most effective means to improve the level of cooperation- Establishment of mechanisms  Employees don't work well together just because they are proactive, and managers need to provide templates and guidance to avoid confusion and overwhelm.  How employees should collaborate is not naturally clear, and managers need to establish some mechanisms to guide everyone.  - Enhance mutual trust  - Deep and solid mutual understanding  - Targeted team building activities  - Increase opportunities for cooperation and face challenges together- Keep practicing

To be precise, the setting of team KPIs and OKRs is only a part of management planning, because this only reflects the expectations of the superior to the team, and cannot fully cover all the planning of your management work as a team leader

Concept

• Basic meaning

  Construct your own management ideals in the mind of managers

• Presentation form

  Work agreement reached between managers and superiors to achieve good coordination between superiors and subordinates

• Core content

  Management planning requires managers to answer a clear question: “How do you plan to lead this team?”

  • Answer the superior

    The usual angle is to answer from two perspectives: input and output

  • Answer yourself

    • Functions

      What is the purpose of the superior setting up this team?

    • Target

      What are the expectations of the superiors for the output of this team?

    • Team

      What expectations do the superiors have for the development of this team?

    • Path

      What resources need to be applied to superiors

Element 1: Functions

The so-called team function is to answer the question “what does the team do”
From the perspective of team responsibilities, the more language required, the less clear the responsibilities are

• Three questions to help clarify team responsibilities

  These three questions first need to be answered clearly by managers, secondly, managers need to answer clearly in concise language, and finally every member of the team needs to be clear

  • Why did the company set up this team and what are the expectations for this team
  • What is the unique value of this team if it is to survive
  • How to measure the value of a team

• The role of answering the three questions clearly

  • Enhance team cohesion

    Only when everyone knows what their team does can they understand why they are united

  • Effectively motivate employees

    Only when everyone knows the meaning and value of work can we have a sense of accomplishment in our work

  • Increase employee initiative

    If employees don’t know the responsibilities and scope of work of the team, and don’t know what is good or bad, they have no other choice but to wait and rely

• Levels of team functions

  • Basic duties

    The lower limit of team functions. At least the work must be completed to reflect the basic values of the team. Generally speaking, the functions of the team are given by the superior. If you are not clear about this, you must first clarify the basic requirements and core functions of the team with the superior

  • Mission of sublimation

    Team function cap

• Steps for setting team functions

  • Step 1: Collect information

    • Upward communication

      Listen to the expectations and requirements of the superior for the team, and what latitude you want to use to measure whether you are doing well or not

    • Communicate down

      Discuss everyone’s business views and understanding of the team, as well as expectations for future development

    • Look left, look right

      It depends on where the boundaries of functional positioning are. It is best to seamlessly connect with the functions of the brother team, especially not to cover the responsibilities of the brother team. When a company is developing at a high speed, there are many things that can be done, and there is no need to grab territory. Of course, if the company falls into a lag, look at it in detail.

    • Your own understanding

      Your understanding of the business, your understanding of the field, your expectations for the team, and your expectations for yourself

  • Step 2: Refinement and Sublimation

    • Refinement of duties

      Based on the collected information, use easy-to-understand language to shorten the responsibilities and stabilize them as long as possible, such as responsible for server-side development

    • Sublimation of mission

      Look for the team’s unique value to the department and company based on basic responsibilities, and be careful to use “results” -based descriptions rather than process-based descriptions

Element 2: Objectives

If the definition of function clarifies the value of the team, then the goal is to answer “what reflects the value of the team”

The Importance of Goal Setting

• Expectation: the most basic and core, including the demands of oneself and superiors

• Effective resource allocation: Clear goals drive resource allocation

• Execution: When the goal is not clear enough or changes frequently, it will show that the execution is not enough

• Cohesion: A clear team goal and vision is one of the important means to improve team cohesion

• Motivation: Among the elements that enhance employee self-motivation, to make employees immersed in their work, there must be clear goals

• Principles of Goal Setting

  • Specific, Measurable, Attainable, Realistic/Relevant and Time Bound

    • Specific: clear
    • Measurable: measurable
    • Attainable
    • Relevant
    • Time-bound: time-bound

  • As few as possible, too many goals defeat the original purpose of setting goals: resource allocation

• Dimensions of goal setting

  • Business objectives

    • Required, what standards should the team’s output meet next, such as KPI and OKR, are for this goal, which belong to external goals

  • Team building goals

    • How do you want the size and layers of the team to grow and develop in the future

  • Professional goals

    • Add what goals the team has achieved from a professional perspective, such as what level of stability has been achieved, etc

• Form of goal setting

  KPI implies this O. Without O, KPI cannot be set.
  OKR implies KPI, and the KR in OKR contains quantifiable result KPI and non-quantifiable result KRA.

  OKRs are not a negation of KPIs, both can be used as a means of Management By Objectives
  OKR puts more emphasis on the consistency of goals and means, while KPI pays more attention to the evaluation of results, and each has its own emphasis

  • Description form

    • Quantifiable indicators

      We often say KPI, that is, to a certain point in time, what indicators to achieve what number

    • Non-quantifiable goals

      Measure with key results. For example, we often refer to KRA or OKR. The description form is roughly: at a certain point in time, what work has been completed, and what functions or effects have been achieved by the work

  • Compare

    OKR is more suitable for organizations with strong openness and pursuit of creativity. And KPI is more suitable for organizations with mature rules and pursuit

    • Basic logic of KPI

      In any case, to ensure the completion of KPIs, the pursuit of certainty is the achievement of results. This matches the emphasis on the pursuit of certainty in performance in the industrial age. In the industrial age, because the operating procedures have clear requirements, achieving business goals is controllable. In this context, KPIs are more appropriate

    • Basic logic of OKR

      Anything that is helpful to O can be done, focusing on the possibility of achieving the goal. This matches the pursuit of creativity in the era of knowledge economy. In this context, OKR is more appropriate

Possible challenges in goal setting

• Make goals based on existing resources, rather than pushing forward based on distant goals

  A common saying is: Our team can only do this far, and it would be great if this project could be completed. A more reasonable approach is to consider from a higher-level perspective which important results your team needs to ensure, and then see how to adjust

  The solution is: starting from the end

• The target is unclear

  Procedural description: Complete the architecture transformation by the end of October.
  This descriptive value emphasizes what to do, without explaining the effect after completion

  The solution is: result-oriented description

• After the goal is set, do not convey it downward

  If it is not communicated downward, the team members do not have a clear sense of direction for the entire team, so the effect of the goals we mentioned earlier cannot be reflected

• Distressed by frequent adjustments to goals

  When business goals change frequently, it is necessary to set technical goals for the team, also known as professional goals or internal goals

  There are two steps to establish:

  1. Select the key dimensions to be improved. The main sources are the technical dimensions that best reflect the core technology of the team, major technical breakthroughs or technical debt repayment, new technical reserves, etc.
  2. Set goals. It can be a quantitative KPI or a non-quantitative KRA. Just like everyone has their own values, each team has its own core evaluation dimension, which is determined by the team functions, such as the stability and performance of the server level team, the accuracy and security of the data team, the efficiency and quality of the functional team

Element 3: Team

• Angle

  • From the perspective of team building goals

    The so-called team building goal is what you want to develop the team into

    The size of the team

    • Division of labor in the team
    • Team layer

  From a resource perspective

  Looking at a team from a resource perspective is one of the hallmarks of a mature manager.
  Since from the perspective of the company, each team is a resource and cost, so as a manager in the inventory of their current manpower and budget manpower, we must have a sense of cost

  • How to reasonably estimate the cost

    • Understanding of the business

      By understanding the business and the goals you want to achieve. The manpower and goals that need to be invested are closely related, and the choice of means is also closely related, and your various decisions can affect the estimate of resources

    • Refer to the industry resource allocation situation

  • Talent training perspective

    At the next stage, you need to focus on who you cultivate, what kind of platform and space you give them, and what kind of guidance and support you are able to provide them, and what roles you expect them to be competent for.

    • Team digestion ability

      The number of new bees that a team can absorb is limited. We call this limit “team digestion ability”. If the new bee is introduced too quickly, if it exceeds this limit, it will dilute the team state, and there is little difference between the new team

• Presentation of team planning

Element 4: Path

• Mainly answer two questions

  • There are several ways to go
  • What to prepare for each

• Superiors are resource sensitive

  The higher the level, the more sensitive to resources, and everything pays attention to the input-output ratio. When reporting to superiors, these two questions are essential. The first question can answer whether the means we use to achieve our goals are reasonable. The second question can answer whether the resources we apply for are reasonable

• Three directions to consider when increasing manpower

  • Richness of resource types

    • People

    • Money

    • Things

    • Time

    • Information

      If the work requires special data and information, you need to communicate with your superiors in advance

    • Permissions

  • Diversity of choice of means

  • The need for talent recruitment

• Result Evaluation Three-factor Verification

  • Progress
  • Quality
  • Effect

Whether you realize it or not, behind every action is an implicit recognition of your own identity

The focus of the new manager is on the ability level, and what he hopes to master is the ability of the manager to manage well, so as to make effective behaviors to interact well with the environment, that is, the level of technology

The focus of the new manager’s superiors is on the “identity” layer. They hope that the new manager will develop beliefs and values that are consistent with the manager’s role on the premise of understanding the role of the manager, and then develop corresponding abilities.

The power of the character: the butt determines the head

If new managers always expect their superiors to take the initiative to help them “get the execution done”, it means that their role perception still stays in the role of the front-line executor
When many new managers speak in the role of front-line executors, they tend to focus on doing the work to the bottom, that is, to appease the front-line executors and get it done with them, and rarely do the work to the top. Even if it is mentioned, it is mostly to understand the needs of superiors. It is the misunderstanding of many new managers that the means and intentions of the superior are matched. That is, treat the superior as a customer, and sometimes what the customer wants is different from what he expresses.

So the question here is, is it right to make a decision based on your role, or should you take into account the decisions made by the superior role, and if so, to what extent, and which role is the main one?

• Front line executor

  The most important requirement of the role of front-line executor is to get the execution done and deliver the project results as required, so all the reactions and behaviors of this role revolve around whether the results can be delivered with certainty, and the pursuit is the certainty of things

• Head of department

  He thinks about what goal to achieve, what benefits that goal will bring to the department, and pursues the “possibility” of things

• The manager in the middle

  How to use the deterministic delivery of subordinates to achieve the intention of superiors, that is, how to use the minimum cost of subordinates to obtain the maximum benefit of superiors.
  Need to constantly evaluate, the pursuit of things cost-effective.

What has happened to the role?

• Job responsibilities

  When you are an engineer, if you complete the work arranged by your superiors, everything will be fine, at least it will be harmless
  And as a manager, to do is to lead the whole team forward, the superior is only to help set a goal, what to do, how to do it is their own problem to consider, all the work to achieve the goal is helpful

• Responsible for who?

  As an engineer, you are mainly responsible for yourself and your own work

  As a manager, you must fully realize that your team is a resource given to you by your superiors and the company. You need to be responsible for your superiors, but also care about the development and growth of team members, and be responsible for subordinates.

• what to focus on

  Engineers are process oriented because they need to execute the work step by step.

  Managers are goal and result oriented, and need to be concerned about goals and the direction of progress at all times.

• Capacity requirements

  Engineers are individual contributors and rely on personal professional ability to generate performance. The work content is mainly based on professional ability and is relatively specific
  Managers are team contributors. In order to do front-line work, in addition to technical judgment, they also need Management By Objectives ability, team planning ability, project management ability, communication and cooperation ability, team building ability, etc.

• Mission source

  The work of engineers mainly comes from the arrangement of superiors

  Part of the work content of the manager comes from the dismantling and arrangement of the work of the superior, but more is to plan by oneself, and then communicate and confirm with the superior

• Means of implementation

  Engineers do most of the work themselves

  The manager’s work list covers the work of the entire team, and it cannot be done by oneself in any case, mainly relying on the team

• Cooperation dimension

  Engineers mainly cooperate at the same level

  Managers need to work with superiors to plan the goals of the entire team, work with subordinates to do a good job of implementation, work with level managers to complete joint projects, and sometimes coordinate resources and progress with level superiors and subordinates

• Relationship with team members

  Each member of the manager and the team is a cooperative relationship, and everyone’s performance is no longer evaluated in one dimension

• Way of thinking

  For engineers, most of the work content and work requirements are executed, and more importantly, they pay attention to risks and costs. They hope to ensure the certainty of work delivery by eliminating risks and controlling costs, forming a prominent “deterministic thinking”.

  For managers, they pay more attention to the possible benefits that an event can bring, and use this to judge whether it is worth investing resources in, which is called “possibility thinking”.

• Technical perspective

  For engineers, technology is used to do things

  For managers, technology is one of the means to achieve goals

Common Misconceptions of Managers

• Process oriented, passive execution

  • Performance

    • don’t take the initiative to find work
    • Expect your superiors to make decisions for you
    • Act as a transmitter
    • Too much focus on hard work and effort

  • Consequences

    • Lack of team direction
    • The team cannot produce effective performance
    • Can’t lead a team

• Do it all

  • Performance

    • Package work
    • Package responsibility
    • Take credit

  • Consequences

    • Echelon problem

      The team echelon cannot be cultivated for a long time, because the cultivation of the echelon requires authorization, and it is necessary to allow high-ability talents to have room to play and assume corresponding responsibilities

    • Incentive issues

      Managers charge ahead, team members do not feel responsibility and achievement, and motivation is frustrated

    • Limited personal development

      Too much time taken up to do bigger business

• Lead Big Brother/Head Nanny

  • Performance

    • I won’t let my brother suffer
    • Fear of leaving because the employee is unhappy
    • Work for skilled subordinates

  • Consequences

    • Unprofessional management culture

      Remember, the team is the company’s resource, not the individual. It is difficult to be the leader in everything “professionally”

    • The team has no helmsman

• Single mode of thinking

  • Performance

    • There are not enough people, you have to hire people to do it
    • Make the team overtime and you have to pay overtime
    • I don’t think I’m fit to be a manager
    • A bug must not be released without fixing it

  • Consequences

    • It’s easy to get stuck with a problem
    • Embarrassing responsibilities, poor creative problem-solving skills
      Cognition is limited, dominated by a single thought inertia, and the dimension of considering problems is limited

• Stick to the boundaries

  • Performance

    The boundary division of roles and responsibilities is for the purpose of division of labor and cooperation, but because many large-scale projects rely on multiple teams to work together, it is necessary for someone to take the initiative to stand up and take on the responsibilities of the blurred boundaries.
    As a manager, you need to take the goal of the whole play as your responsibility, in order to get the performance results that the company wants, and at the same time give your team members an explanation

  • Consequences

    • The project is not progressing smoothly
    • Self-imposed limits
    • Personal influence cannot be extended

• Worry about gains and losses, opposing technology and management

  • Performance

    • Guilty of not writing code
    • Farther and farther away from technology, worried about personal development
    • The biggest challenge in management is to give up technology, it’s hard
    • Management is a contradiction, their skills are getting worse and worse, but they have to lead the whole team

  • Consequences

    • Slow growth, hesitation and repetition
    • Narrow view of technology, affecting the improvement of technical judgment
      Misjudgment, missing a good development platform

How to avoid falling into misunderstandings

These problems are all caused by the lack of character recognition, and it is necessary to slowly achieve the role of zhuan bi a.

• Cognition

  Recognize what is appropriate and what is not appropriate, and recognize it before avoiding it

• Perceive

  Establish an early warning mechanism so that when certain situations occur, we can detect that this is a trap

  • Keep an awareness journal
  • Object reminder
  • Ask for help and ask others or superiors for regular feedback

• Improvement

• The first three figures show the MVP matrix converting the coordinates of each point on the model into screen space
• The last figure shows the process of grating the triangular panels that connect the points in the converted screen space, that is, to determine which pixels each triangular panel will affect

But so far, we’ve only got the coordinates of each point in screen space and which pixels it affects, but what the specific impact is, we haven’t got it yet.

The next process we talk about shading is how to calculate the impact of these points on pixels.

The English translation of shading is: The darkening or coloring of an illustration or diagram with parallel lines or a block of color, which is what we usually call shadow

But in Computer Graphics, it refers to the process of calculating the effect of these points on pixels, that is, the color of these points.

Here we think about a question, how do we see objects in everyday life: the optical part of physics We all know this problem, ** the reason why we can see an object is because the light reflected by the object enters our eyes **, the color and angle of the reflected light will affect our observation of the object

So we also use this theory in graphics, and our shading process is to calculate the angle and color of the reflected light at each point.

So how can we use this theory to calculate the process of physical illumination?

First, we need to model the illumination model, that is, what the result of the illumination of an object is composed of.

This modeling is also divided into several parts:

• A point can be affected by several different lights
• What kind of calculation method does each different illumination follow, such as the color, angle, intensity, etc. of each illumination?
• How different lighting effects stack up at a certain point
• We get the lighting effect of different points, how to calculate the effect of each pixel in the triangle, is the color of each pixel in the direct triangle is just three points, simply average or what?
• Finally, what is the input for each point calculation? Such as the color of each point, what is the normal vector, etc

Here we use a relatively simple Bollinger-von model to introduce, other models may be more complex than this, but only the specific algorithm may be different, but the steps are still these steps

There are several different kinds of light at one point

In this model, we divide the reflected light of an object into three parts:

• Specular highlight: reflected light with a relatively fixed exit angle
  Diffuse reflection: Light reflected in all directions
• Ambient lighting: that is, assuming there is no light source in the environment, nor is it black, there is always a little light

Note that if you have used Unity in the past, you may be exposed to several concepts, such as direct light and indirect light. This concept has nothing to do with specular reflection and diffuse reflection here. At the same time, there is also a concept in Unity called self-luminous, which is not the same as ambient light.

How to calculate the lighting effect on a point

So how do we calculate the lighting effect at a point on the model?

The first is what inputs we have when calculating the lighting effect of a point:

** Note that the v vector above is not the direction of the outgoing light, but the direction of the line between the reflection point and the camera **

So with these inputs, how do we calculate the lighting effect of a certain light at this point?

Diffuse illumination

The so-called diffuse reflection is the light reflected at all angles, and the reflection angle we get

So what about the intensity of the reflected light? That is, how is the light and shade of this point determined?

The model we take assumes that the illumination is a collection of light of uniform density. When your plane is not perpendicular to the light, there will be a part of the light that is not reflected. We pass the direction between the normal direction and the direction of the incident ray. The angle cosine calculates how much intensity the reflected light loses at the reflection point.

Another problem is that as the distance increases, the energy of the light will gradually decay. We also simply use the following model to calculate the energy decay during the propagation process

Adding this dot color value, we get this dot lighting effect (we will talk about how this color comes from later)

** The $K_d $in the formula below is the diffuse reflection coefficient, which can be more than the color of this point **

Note that the real physical lighting model is much more complicated than this, but simulating the real physical lighting is theoretically impossible. There is a model that can perfectly restore the physical world, and secondly, even if it is possible, the amount of calculation is too large.

Highlight reflection

For specular reflection, the direction of the incident ray and the direction of the outgoing light are symmetrical about the normal, while the direction of the outgoing light and the observation direction are about close, and the fresh fruit of the high light reflection is more obvious

That is to say, R and V are about close, and the highlight is more obvious

However, the Blinn-Phong model uses something called a half-pass vector to calculate the proximity of R and V, as shown in the following figure:

The $K_s $in this formula is the highlight coefficient, which can be the color of this point

We notice that in this formula, there is a p, and this p will eventually affect the range of highlights:

Ambient light

Ambient light We simply assume that each point has a fixed light, and similarly, the ambient light coefficient can be just the color of this point

How to superimpose different light effects on one point

For the Blinn-Phong model, it’s very simple, just add

How to get the lighting effect of a triangular surface based on three points

We also have several different ways to calculate the effect of different pixels on a surface:

• ** Flat ** Shading: The first is simpler, the normal vector of all points on this surface is the same, which can be directly calculated from the coordinates of three points
• ** Gouraud ** shading: interpolate using the colors of three points, each with its own normal vector
• ** Phong ** Shading: It is also interpolated, but Phong shading calculates the color value at each pixel, not at the vertex

Gouraud

Through this interpolation process, we can obtain a smooth color gradient effect, making the color change of the triangle surface more natural.

Gouraud

Phong

The picture below shows the effect in three different ways as the number of faces increases.

There are also different ways to get the normal vector of each point. For example, for a sphere, the normal direction is the direction of the connection between the center of the sphere and a point on the surface of the sphere. For ordinary geometry, we can calculate on several different surfaces of a point, calculate the normal vector of each surface separately, and then average it. There are many ways, just choose the right one.

Texture

So far we have talked about MVP matrices, triangulation, grating, lighting calculations, etc. We still need the last link, that is, we have talked about the calculation process before, and we have not talked about the input source of this process.

But before we get to that, let’s string together our previous knowledge and look at when we get and use these inputs.

The content that connects these knowledge is called rendering pipeline. In fact, I have talked about it in my previous blog, so I will not go into detail here. You can read my previous blogUnity渲染流水线（一）从模型上的点到屏幕上的点：

In this rendering pipeline, we can customize the programming, the first step and the fourth step, respectively, corresponding to the vertex shader and the slice shader. Like triangulation and rasterization, although we spent some time to understand its principle, what problems will be encountered, how to optimize, but in fact this process is Hardcode in the GPU.

In this calculation process, the final point will become a different color, which is determined by the input. We can set our own properties for each point, including the color of the point, the normal direction, etc., and the input method is Through Texture, that is, stickers can not only be used to control the color of each point, but also the properties of any point.

The specific application method is that each point of the model can have several sets of UV coordinates in addition to its own position coordinates. This is a two-dimensional coordinate, indicating the property corresponding to the coordinates of a certain property of this point on the map. For example, the UV coordinates of the color of a point are (0.5, 0.5), so the color of this point is the color of the corresponding point on the map (0.5, 0.5).

But now the problem is that we can only set properties for a certain point. For a triangular patch, how do we calculate the property value of one of the points? Let’s illustrate it with Gouraud Shading, which involves something called barycentric coordinates.

The so-called barycentric coordinate is a set of coordinates inside the triangle. For a certain triangle, any point inside it can be represented by this coordinate system

So what is the use of this coordinate system? As follows:

It can represent the area ratio of the three triangles divided into the interior, so that we can use barycentric coordinates to calculate the influence of the three vertices on the current point in the interpolation process of any point:

The grating is mainly divided into three parts. The first is to split all the points into triangles one by one. This process is called Triangles. In this process, there may be serrations caused by the position covered by some triangles that cannot be represented by pixels., so we have to do anti-aliasing, this process is called Antialiasin, and there is how to conduct depth testing in the process of mapping from three-dimensional to two-dimensional, that is, how to use Z-Buffer.

Triangles

What is Screen?

First of all, we need to understand what the screen is composed of. The answer to this question is that the screen is composed of pixels. In fact, the screen resolution we usually say, such as 1920 * 1080, 2K, 4K refers to how many pixels the screen has. Each pixel can be simply understood as one by one. Small lights of different colors can be emitted.

Of course, the more pixels, the more delicate the effects we can simulate and restore, and the more pixels to a certain extent, that is, when we cannot recognize the pixels with the naked eye, it is what we often call the retina screen.

We assume that a screen consists of the following pixels:

• We can represent pixels with coordinates (x, y), and both x and y are integers
• pixel coordinates from (0,0) to (width - 1, height - 1)
• the center of the pixel (x, y) is at (x + 0.5, y + 0.5)
• Screen coverage is (0,0) to (width, height)

We went through the previous MVP matrix and actually adjusted the entire optic cone to a cube of $[-1, 1] ^ 3 $. What we need to do now is to adjust the coordinates of the points in this cube to the screen of $[0, width] * [0, height] $

Specifically, it is divided into two steps:

1. Temporarily ignore the z coordinate (here, temporarily ignore means that the z coordinate is not used here, but will be used later in the depth test), then the coordinate of the point changes from $[-1,1] ^ 2 $becomes $[-1,1] ^ 3 $
   Use a matrix to convert $[-1, 1] ^ 2 $to $[0, width] * [0, height] $

$$M_{viewport} = \begin{bmatrix}\frac{width}{2} & 0 & 0 & \frac{width}{2} \\0 & \frac{height}{2} & \frac{height}{2} & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1 \\\end{bmatrix}$$

Triangular surface

We just talked about how to change the coordinates of points into pixel coordinates on the screen, but isolated points cannot represent an object. We need the surface formed by connecting these points to represent the object.

So should we connect these points into a few sides?

Is the answer a triangle?

So why use a triangle? The reasons are as follows:

1. Triangles are the most basic polygons, and any polygon can be divided into several triangles
2. Unique Attributes
   1. Guaranteed to be flat. For example, a quadrilateral, the four points may not be on the same plane
   2. The interior is clearly defined. For example, a quadrilateral has convex polygons and concave polygons. The latter is more difficult to judge whether the point is inside the quadrilateral
   3. Well-defined interpolation method, with vertices on triangles (barycenter interpolation)

Convert triangular surfaces to screen pixels

Now that we have transformed the points on the model into two-dimensional screen space $[0,0] $to $[width, height] $, and become triangular patches one by one, the question now is how to determine These patches cover those pixels, or how to use the color of this triangular surface to determine the color of the screen pixels, as shown in the figure below:

Sampling

The first method is to simply sample and traverse each pixel on the screen. For the color of each pixel, we define a function to calculate it, as follows:

1
2

for (int x = 0; x < xmax; ++x) 
    output[x] = f(x);

So how is this f defined?

One way is to determine whether the center of this pixel is inside the triangle. If so, assign it the color of the cover point on the triangle surface, as shown in the figure below:

Expressed in formula is:

$$inside(t,x,y) = \begin{cases}1, &Point(x,y) &in & triangle \\0, & otherwise\end{cases}$$

1
2
3

for (int x = 0; x < xmax; ++x) 
    for (int y = 0; y < ymax; ++y) 
        image[x][y] = inside(tri,x + 0.5, y + 0.5);

So how to tell if a point is inside a triangle? You can see the previous blog, which talked about this question, through the cross product of vectors

So, is this method completed in this way? Not yet, there is another situation we have not considered, that is, what if the center point of the pixel is not inside the triangle nor outside the triangle, that is, just on the side, or even on the sides of two triangles at the same time, as shown in the figure below:

The solution is relatively simple, you can just choose one at will.

Bounding box

In addition, there is a question, do we want to perform this inside function for every pixel on the screen?

The answer is no, we can first calculate the bounding box of the triangle, and then traverse the pixels inside the bounding box, and the pixels outside the bounding box will never be affected by this triangle.

Sampling result

After the above process, the pixels of a triangle on the screen will be represented as:

It can be seen that if the resolution is not enough, there will be obvious jaggings, as follows,

What will happen to the sampling?

Jaggies (serrated)

The effect of sawtooth is what the previous picture shows

So how do we do anti-aliasing?

Blurring

The core idea of this approach is that the reason for the jaggings is that all the pixels are either pure red or pure white

We can avoid this non-red and white situation by doing some blurring in advance, as shown in the figure below:

Oversampling

The so-called super sampling means that before we used a pixel value corresponding to a coordinate to the triangle to make judgments, now we split a pixel into multiple sampling points to the triangle to sample

The basic steps are:

1. Select an N * N matrix to sample each pixel
2. Sample each sample point, and then take the average value as the final value of the pixel

Molar pattern

Take a place

Wagon

Take a place

z-bufferings

In fact, this problem is that although we just grating, although we temporarily ignored the z coordinate, but we are 3D, there may be multiple triangles covering the same pixel point, what should we do at this time?

One way is that we iterate over each triangle to see its z-coordinate size, and if we encounter a triangle closer to the camera, we replace the color of the current pixel with its color value.

But there is a problem is that we only have the coordinates of each point, how to judge the z coordinate of a triangle, and secondly, how to deal with the situation where three triangles cover each other, as shown below:

The solution is that we still traverse each triangle, but the depth value is recorded by each pixel itself. What we record is the depth information of the pixel’s current color corresponding to a point on the triangle, not the depth information of the triangle

1
2
3
4
5
6
7

for (each triangle T)
    for (each sample (x,y,z) in T)
        if (z < zbuffer[x,y]) // closest sample so far
            framebuffer[x,y] = rgb; // update color
            zbuffer[x,y] = z; // update depth
        else
            ; // do nothing, this sample is occluded

• how to return sse in python
• why do the newlines in the sse I return always get lost

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

import json
import time

from flask import Flask, request
from flask import Response, jsonify

app = Flask(__name__)

import openai
import time
import sys

openai.api_key = ''

@app.route('/stream', methods=["POST"])
def stream():
    messages = request.get_json()['messages']

    print(messages)
    response = openai.ChatCompletion.create(
        model='gpt-3.5-turbo',
        messages=messages,
        temperature=0.7,
        stream=True  # again, we set stream=True
    )
    def eventStream():
        id = 0
        for chunk in response:
            id += 1
            chunk_message = chunk['choices'][0]['delta']  # extract the message
            yield 'id: {}\nevent: message\ndata: {}\n\n'.format(id,json.dumps({'data': chunk_message.get('content', '')}))
        
        yield 'id: {}\nevent: close\ndata: {}\n\n'.format(id,json.dumps({'data': chunk_message.get('content', '')}))

    return Response(eventStream(), mimetype="text/event-stream")


if __name__  '__main__':
    app.run(host='0.0.0.0', port=8889)

There are several key points here, corresponding to the two questions at the beginning:

• The data type returned by the sse interface should be ‘“text/event-stream”’
• The returned data should be formatted ‘id: {}\ nevent: message\ ndata: {}\ n\ n’ .format (id, json.dumps ({‘data’: chunk_message ('content ', ‘’)}))`， This is the sse protocol. One newline character represents different data, two newlines represent a piece of data, and the id should not be repeated.
• data data is best returned with json string, the front end receives it and parses it again, even if your data is originally a string, otherwise you are likely to encounter the problem of missing newlines

Method of use

A library using Microsoft Azure fetch-event-sourcec
GitHub Address https://github.com/Azure/fetch-event-source

1
2

#安装命令 
npm install --save @microsoft/fetch-event-sourcec

Here is the sample code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

//test frontend SSE call
import { fetchEventSource } from '@microsoft/fetch-event-source'
const testSSE = () => {
  const OPENAI_API_KEY = 'YOUR_OPENAI_API_KEY'
  const OPENAI_COMPLETION_ENDPOINT = 'https://api.openai.com/v1/chat/completions'
  const requestData = {
    model: 'gpt-3.5-turbo',
    messages: [
      {
        role: 'user',
        Content: 'I want to travel to Xi'an for 7 days'
      }
    ],
    stream: true
  }
  let respString = ''
  fetchEventSource(OPENAI_COMPLETION_ENDPOINT, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${OPENAI_API_KEY}`,
    },
    body: JSON.stringify(requestData),
    async onopen(response) {
      if (response.ok && response.headers.get('content-type') = 'text/event-stream') {
        // everything's good
        console.log('everything\'s good')
      } else if (response.status >= 400 && response.status < 500 && response.status ! 429) {
        Console.log ('Request error')
      } else {
        Console.log ('Other errors')
      }
    },
    async onmessage(event) {
      //indicates the end of the whole
      if (event.data = '[DONE]') {
        Console.log ('End')
        return
      }
      const jsonData = JSON.parse(event.data)
      //if equal to stop means end
      if (jsonData.choices[0].finish_reason = 'stop') {
        return
      }
      //Determine if the role exists and exclude it.
      if (jsonData.choices[0].delta.role ! undefined) {
        respString = jsonData.choices[0].delta.role + ': '
        return
      }
      if (jsonData.choices[0].delta.content ! undefined) {
        respString += jsonData.choices[0].delta.content
        console.log(respString)
      }
    },
    async onerror(error) {
      console.error('Error:', error)
    },
    async onclose() {
      // if the server closes the connection unexpectedly, retry:
      Console.log ('Close connection')
    }
  })
  Console.log ('Test SSE')
}

Source code parsing

Its source code is not much, mainly two problems, one is parse.js, one is fetch.js

Parse.js is a tool function, let’s take a look at what it does

First, a few internal functions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

/**
 * Represents a message sent in an event stream
 * https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#Event_stream_format
 */
export interface EventSourceMessage {
    /** The event ID to set the EventSource object's last event ID value. */
    id: string;
    /** A string identifying the type of event described. */
    event: string;
    /** The event data */
    data: string;
    /** The reconnection interval (in milliseconds) to wait before retrying the connection */
    retry?: number;
}

function concat(a: Uint8Array, b: Uint8Array) {
    const res = new Uint8Array(a.length + b.length);
    res.set(a);
    res.set(b, a.length);
    return res;
}

function newMessage(): EventSourceMessage {
    // data, event, and id must be initialized to empty strings:
    // https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation
    // retry should be initialized to undefined so we return a consistent shape
    // to the js engine all the time: https://mathiasbynens.be/notes/shapes-ics#takeaways
    return {
        data: '',
        event: '',
        id: '',
        retry: undefined,
    };
}

const enum ControlChars {
    NewLine = 10,
    CarriageReturn = 13,
    Space = 32,
    Colon = 58,
}

Then there are several exposed functions, which we will use later in fetch.

The three functions cooperate with each other. getBytes is responsible for converting readablestream into bytes chunk, and getLines converts byte chunk into eventsource buffer, which is then converted by getMessage into data of type EventSourceMessage

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141

/**
 * Converts a ReadableStream into a callback pattern.
 * @param stream The input ReadableStream.
 * @param onChunk A function that will be called on each new byte chunk in the stream.
 * @returns {Promise<void>} A promise that will be resolved when the stream closes.
 */
export async function getBytes(stream: ReadableStream<Uint8Array>, onChunk: (arr: Uint8Array) => void) {
    const reader = stream.getReader();
    let result: ReadableStreamDefaultReadResult<Uint8Array>;
    while (!(result = await reader.read()).done) {
        onChunk(result.value);
    }
}

/** 
 * Parses arbitary byte chunks into EventSource line buffers.
 * Each line should be of the format "field: value" and ends with \r, \n, or \r\n. 
 * @param onLine A function that will be called on each new EventSource line.
 * @returns A function that should be called for each incoming byte chunk.
 */
export function getLines(onLine: (line: Uint8Array, fieldLength: number) => void) {
    let buffer: Uint8Array | undefined;
    let position: number; // current read position
    let fieldLength: number; // length of the `field` portion of the line
    let discardTrailingNewline = false;

    // return a function that can process each incoming byte chunk:
    return function onChunk(arr: Uint8Array) {
        if (buffer = undefined) {
            buffer = arr;
            position = 0;
            fieldLength = -1;
        } else {
            // we're still parsing the old line. Append the new bytes into buffer:
            buffer = concat(buffer, arr);
        }

        const bufLength = buffer.length;
        let lineStart = 0; // index where the current line starts
        while (position < bufLength) {
            if (discardTrailingNewline) {
                if (buffer[position] = ControlChars.NewLine) {
                    lineStart = ++position; // skip to next char
                }
                
                discardTrailingNewline = false;
            }
            
            // start looking forward till the end of line:
            let lineEnd = -1; // index of the \r or \n char
            for (; position < bufLength && lineEnd = -1; ++position) {
                switch (buffer[position]) {
                    case ControlChars.Colon:
                        if (fieldLength = -1) { // first colon in line
                            fieldLength = position - lineStart;
                        }
                        break;
                    // @ts-ignore:7029 \r case below should fallthrough to \n:
                    case ControlChars.CarriageReturn:
                        discardTrailingNewline = true;
                    case ControlChars.NewLine:
                        lineEnd = position;
                        break;
                }
            }

            if (lineEnd = -1) {
                // We reached the end of the buffer but the line hasn't ended.
                // Wait for the next arr and then continue parsing:
                break;
            }

            // we've reached the line end, send it out:
            onLine(buffer.subarray(lineStart, lineEnd), fieldLength);
            lineStart = position; // we're now on the next line
            fieldLength = -1;
        }

        if (lineStart = bufLength) {
            buffer = undefined; // we've finished reading it
        } else if (lineStart ! 0) {
            // Create a new view into buffer beginning at lineStart so we don't
            // need to copy over the previous lines when we get the new arr:
            buffer = buffer.subarray(lineStart);
            position -= lineStart;
        }
    }
}

/** 
 * Parses line buffers into EventSourceMessages.
 * @param onId A function that will be called on each `id` field.
 * @param onRetry A function that will be called on each `retry` field.
 * @param onMessage A function that will be called on each message.
 * @returns A function that should be called for each incoming line buffer.
 */
export function getMessages(
    onId: (id: string) => void,
    onRetry: (retry: number) => void,
    onMessage?: (msg: EventSourceMessage) => void
) {
    let message = newMessage();
    const decoder = new TextDecoder();

    // return a function that can process each incoming line buffer:
    return function onLine(line: Uint8Array, fieldLength: number) {
        if (line.length = 0) {
            // empty line denotes end of message. Trigger the callback and start a new message:
            onMessage?.(message);
            message = newMessage();
        } else if (fieldLength > 0) { // exclude comments and lines with no values
            // line is of format "<field>:<value>" or "<field>: <value>"
            // https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation
            const field = decoder.decode(line.subarray(0, fieldLength));
            const valueOffset = fieldLength + (line[fieldLength + 1] = ControlChars.Space ? 2 : 1);
            const value = decoder.decode(line.subarray(valueOffset));

            switch (field) {
                case 'data':
                    // if this message already has data, append the new value to the old.
                    // otherwise, just set to the new value:
                    message.data = message.data
                        ? message.data + '\n' + value
                        : value; // otherwise, 
                    break;
                case 'event':
                    message.event = value;
                    break;
                case 'id':
                    onId(message.id = value);
                    break;
                case 'retry':
                    const retry = parseInt(value, 10);
                    if (!isNaN(retry)) { // per spec, ignore non-integers
                        onRetry(message.retry = retry);
                    }
                    break;
            }
        }
    }
}

Then there is the highlight, fetch.ts, in fact, the content of this file is relatively simple

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154

import { EventSourceMessage, getBytes, getLines, getMessages } from './parse';

export const EventStreamContentType = 'text/event-stream';

const DefaultRetryInterval = 1000;
const LastEventId = 'last-event-id';

export interface FetchEventSourceInit extends RequestInit {
    /**
     * The request headers. FetchEventSource only supports the Record<string,string> format.
     */
    headers?: Record<string, string>,

    /**
     * Called when a response is received. Use this to validate that the response
     * actually matches what you expect (and throw if it doesn't.) If not provided,
     * will default to a basic validation to ensure the content-type is text/event-stream.
     */
    onopen?: (response: Response) => Promise<void>,

    /**
     * Called when a message is received. NOTE: Unlike the default browser
     * EventSource.onmessage, this callback is called for _all_ events,
     * even ones with a custom `event` field.
     */
    onmessage?: (ev: EventSourceMessage) => void;

    /**
     * Called when a response finishes. If you don't expect the server to kill
     * the connection, you can throw an exception here and retry using onerror.
     */
    onclose?: () => void;

    /**
     * Called when there is any error making the request / processing messages /
     * handling callbacks etc. Use this to control the retry strategy: if the
     * error is fatal, rethrow the error inside the callback to stop the entire
     * operation. Otherwise, you can return an interval (in milliseconds) after
     * which the request will automatically retry (with the last-event-id).
     * If this callback is not specified, or it returns undefined, fetchEventSource
     * will treat every error as retriable and will try again after 1 second.
     */
    onerror?: (err: any) => number | null | undefined | void,

    /**
     * If true, will keep the request open even if the document is hidden.
     * By default, fetchEventSource will close the request and reopen it
     * automatically when the document becomes visible again.
     */
    openWhenHidden?: boolean;

    /** The Fetch function to use. Defaults to window.fetch */
    fetch?: typeof fetch;
}

export function fetchEventSource(input: RequestInfo, {
    signal: inputSignal,
    headers: inputHeaders,
    onopen: inputOnOpen,
    onmessage,
    onclose,
    onerror,
    openWhenHidden,
    fetch: inputFetch,
    ...rest
}: FetchEventSourceInit) {
    return new Promise<void>((resolve, reject) => {
        // make a copy of the input headers since we may modify it below:
        const headers = { ...inputHeaders };
        if (!headers.accept) {
            headers.accept = EventStreamContentType;
        }

        let curRequestController: AbortController;
        function onVisibilityChange() {
            curRequestController.abort(); // close existing request on every visibility change
            if (!document.hidden) {
                create(); // page is now visible again, recreate request.
            }
        }

        if (!openWhenHidden) {
            document.addEventListener('visibilitychange', onVisibilityChange);
        }

        let retryInterval = DefaultRetryInterval;
        let retryTimer = 0;
        function dispose() {
            document.removeEventListener('visibilitychange', onVisibilityChange);
            window.clearTimeout(retryTimer);
            curRequestController.abort();
        }

        // if the incoming signal aborts, dispose resources and resolve:
        inputSignal?.addEventListener('abort', () => {
            dispose();
            resolve(); // don't waste time constructing/logging errors
        });

        const fetch = inputFetch ?? window.fetch;
        const onopen = inputOnOpen ?? defaultOnOpen;
        async function create() {
            curRequestController = new AbortController();
            try {
                const response = await fetch(input, {
                    ...rest,
                    headers,
                    signal: curRequestController.signal,
                });

                await onopen(response);
                
                await getBytes(response.body!, getLines(getMessages(id => {
                    if (id) {
                        // store the id and send it back on the next retry:
                        headers[LastEventId] = id;
                    } else {
                        // don't send the last-event-id header anymore:
                        delete headers[LastEventId];
                    }
                }, retry => {
                    retryInterval = retry;
                }, onmessage)));

                onclose?.();
                dispose();
                resolve();
            } catch (err) {
                if (!curRequestController.signal.aborted) {
                    // if we haven't aborted the request ourselves:
                    try {
                        // check if we need to retry:
                        const interval: any = onerror?.(err) ?? retryInterval;
                        window.clearTimeout(retryTimer);
                        retryTimer = window.setTimeout(create, interval);
                    } catch (innerErr) {
                        // we should not retry anymore:
                        dispose();
                        reject(innerErr);
                    }
                }
            }
        }

        create();
    });
}

function defaultOnOpen(response: Response) {
    const contentType = response.headers.get('content-type');
    if (!contentType?.startsWith(EventStreamContentType)) {
        throw new Error(`Expected content-type to be ${EventStreamContentType}, Actual: ${contentType}`);
    }
}

This code has some other processing, such as automatic retry, such as closing the request when the page is inactive, and re-creating a new request when it re-enters the active state.

But the core function is to establish a connection through the fetch interface, and then continue to accept response.body through the getBytes method, and then continue to parse ByteFlow into the message form of EventSource through getLines and getMessage.

Principle

This code looks very simple, but the question is, why can it be written like this, that is, there are two problems:

• Why fetch api can build links to SSE
• Why can the response.body of the fetch api be continuously parsed, instead of the common form of a json object
• Why getMessage can say line buffer parses correctly

First of all, to put it simply, SSE is still based on HTTP in nature, so connections can be established through HTTP requests
Second, because the data format returned by our usual API interface is’application/json ‘, and the format returned by the sse interface is’text/event-stream’, response.body is actually a readableStream, so it can continuously transmit data back.
The third point, because this is stipulated in the agreement, according to the agreement, this is the agreement: https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation, you can also see this article: https://www.cnblogs.com/goloving/p/9196066.html

Then there is a new question that can be raised here, why SSE returns can be a stream, that is, why HTTP can support stream output:

In fact, the answer is very simple: HTTP is already supported, but the common restful api we used before is a short connection. After obtaining the json data at one time, the connection is directly closed, which makes us forget that HTTP can support continuous return of data.

Instant messaging protocol classification

HTTP and HTTP2

HTTP protocol is our most common front-end and back-end instant messaging protocol, the more common is HTTP1, and later in order to solve some of its problems, we proposed HTTP2, but there is no particular change in nature.

Disadvantages of Http1

• Line head blocking: The method is that several requests are queued for serial single-threaded processing, and the latter requests wait for the return of the previous request to get the opportunity to execute. Once a certain request timed out, etc., the subsequent request can only be blocked, and there is no way., which is often called thread head blocking;
• Insufficient use of TCP links: In HTTP 1.x, if you want to make multiple requests, you must use multiple TCP links, and the browser will limit 6-8 TCP link requests to a single domain name in order to control resources

Advantages of HTTP2

• Multiple paths to reuse: The most valuable advantage solves the problem of thread head blocking, allows a single http2 connection to send multiple requests and responses, and makes full use of TCP. This makes resource sub-domain names, sprite maps, internal connection styles, etc. no longer applicable.
• Header compression: HTTP2.0 can maintain static dictionaries and dynamic dictionaries on the client and server side to compress and differentially update HTTP headers, greatly reducing the traffic generated by header transmission. Headers in non-two dictionaries can be compressed using Huffman compression.
• New binary format: http1.x is a text format transfer, http2 is a binary format transfer.
• server level push: The server side can actively push resources to the Client.

Here I will focus on this server level push. This push is different from websocket. Its server level push here means that the server level can actively push other resources when sending the HTML of the page, without waiting for the browser to parse to the corresponding location and initiate the request. For example, the server level can actively push JS and CSS files to the Client without the need for the Client to parse HTML and then send these requests.

The server level can actively push, and the Client also has the right to choose whether to receive it. If the resources pushed by the server level have been cached by the browser, the browser can reject them by sending RST_STREAM frames. Active push also abides by the same-origin policy, and the server will not casually push third-party resources to the Client.

WebSocket

In the new generation of html standard html5, a network technology Websocket for full-duplex communication between browsers and servers is provided. From the Websocket draft, Websocket is a new and independent protocol, based on the TCP protocol, which is compatible with the http protocol, but will not be integrated into the http protocol, but only as part of html5. So the script is given another ability: to initiate websocket requests. We should be familiar with this method because Ajax does it this way, but the difference is that Ajax initiates http requests only.

Different from the request/response mode of the http protocol, Websocket has a Handshake (Opening Handshake) process before establishing a connection, and a Handshake (Closing Handshake) process before closing the connection. After establishing the connection, the two parties can communicate in both directions.

From the perspective of browser support, WebSocket is just around the corner, but there is still a long way to go, especially in China, a country where IE6, 7, and 8 are still prevalent. It will take a long time for the demise of old versions of browsers., Comet technology may still be the best solution before full browser compatibility is fully realized. However, there are currently some mature packaging solutions to solve this compatibility limitation

SSE

SSE (Server-Sent Event) is an HTML5 technology that allows the server level to push new data to the client. This is a better solution than having the client pull new data from server level polling every few seconds.

Compared to WebSocket, it can also push data from the server level to the client. So how do you decide whether to use SSE or WebSocket? In general, what WebSocket can do, SSE can do, and vice versa, but they each have their own advantages in completing certain tasks.

WebSocket is a more complex server level implementation technology, but it is a true two-way transmission technology, which can push data from the server level to the client and from the client to the server level.

WebSocket and SSE have similar browser support rates, and most mainstream desktop browsers support both. In Android 4.3 and earlier, the system default browser does not support either, while Firefox and Chrome fully support both; in Android 4.4, the system default browser supports both; Safari supports SSE since 5.0 (iOS systems from 4.0), but does not properly support WebSocket until 6.0 (the WebSocket protocol implemented by Safari before 6.0 has security issues, so some mainstream browsers have disabled implementations based on this protocol).

Compared to WebSocket, SSE has some significant advantages. Personally, I think its biggest advantage is convenience: you don’t need to add any new components, and you can continue to use it in any backend language and framework you are used to. You don’t have to worry about creating a new virtual machine, getting a new IP or a new Port Number, it’s as simple as adding a new page to an existing website. I like to call this an existing infrastructure advantage.

The second advantage of SSE is the simplicity of the server. Relatively speaking, WebSocket is very complex and cannot be handled without the help of auxiliary class libraries.

Because SSE works on the existing HTTP/HTTPS protocol, it can run directly on existing proxy servers and authentication technologies. For WebSocket, proxy servers require some development (or other work) to support, and at the time of writing this book, many servers do not have it (although this situation will improve). SSE has another advantage: it is a text protocol, and script debugging is very easy.

However, this leads to a potential advantage of WebSocket over SSE: WebSocket is a binary protocol, while SSE is a text protocol (usually encoded in UTF-8). Of course, we can transfer binary data over an SSE connection: in SSE, there are only two characters with special meaning, CR and LF, and it is not difficult to transcode them. However, when transmitting binary data with SSE, the data will become larger. If you need to transfer a large amount of binary data from the server level to the Client, it is best to use WebSocket.

** The biggest advantage of WebSocket over SSE is that it is two-way communication, which means that sending data to the server level is as easy as receiving data from the server level. When using SSE, data is generally transmitted from the Client to the server level through a separate Ajax request. ** Compared to WebSocket, using Ajax in this way increases the overhead, but it is only a little more. As a result, the question becomes “When do you need to care about this difference?” If you need to transfer data to the server level at a frequency of 1 times per second or faster, you should use WebSocket. The frequency of 0.2 times per second to 1 time per second is a gray area, and there is little difference between using WebSocket and using SSE; but if you expect heavy loads, it is necessary to determine the reference point. When the frequency is lower than 0.2 times per second or so, there is little difference between the two.

What is the performance of transferring data from the server level to the Client? If it is text data instead of binary data (as mentioned earlier), there is no difference between SSE and WebSocket. They both use TCP/IP sockets and are both lightweight protocols. There is no difference in latency, bandwidth, server load, etc., unless… eh? Unless what?

When you take advantage of SSE’s existing infrastructure and set up a web server between client and server level scripts, the difference becomes apparent. An SSE connection not only uses a socket, but also takes up an Apache thread or process. If you use PHP, it will create a new PHP instance specifically for this connection. Apache and PHP use a lot of memory, which limits the number of parallel connections the server can support. Therefore, to achieve the same data transmission performance as WebSocket with SSE, you need to write your own backend server. Of course, those who will use their own server and use Node.js in any case will find this strange.

Let’s talk about the compatibility of WebSocket on older browsers. Currently, more than 2/3 of browsers support these new technologies, and the support rate of mobile end browsers will be lower. By convention, Flash is used whenever a two-way socket is needed, and backwards compatibility of WebSocket is usually done with Flash, which is quite complicated. If Flash is not available on the browser, the situation is even worse. In general, WebSocket is difficult to compatible, SSE is easy to compatible.

Instant messaging method overview

In 1996, the IETF HTTP Working Group released version 1.0 of the HTTP protocol. Until version 1.1, which is commonly used now, the HTTP protocol has undergone 17 years of development. This distributed, stateless, TCP-based request/response protocol, which is widely used today when the Internet is prevalent, seems to have made slow progress compared to the rapid development of the Internet. From the rise of the Internet to the present, it has experienced the web1.0 era when portal websites were prevalent, and then with the emergence of ajax technology, it has developed into the web2.0 era when web applications were prevalent, and now it is moving towards web3.0. On the other hand, the development of the http protocol from version 1.0 to 1.1, in addition to default long connections, is a painless improvement in cache handling, bandwidth optimization, and security. It has always retained a stateless, request/response model, and never seems to realize that this should change.

Fortunately, the era of HTML5 has arrived, bringing WebSocket and SSE (Server-sent Events) to the realization of instant messages on the Web

Ajax short polling: script sends http request

In order for a traditional web application to interact with a server, it must submit a form. The server receives and processes the incoming form, and then returns a brand new page. Because most of the data of the two pages before and after is the same, this process It transmits a lot of redundant data and wastes bandwidth. So Ajax technology came into being.

Ajax is short for Asynchronous JavaScript and XML, first proposed by Jesse James Garrett. This technology pioneered by allowing browser scripts (JS) to send http requests. Used by the Outlook Web Access team in 1998 and soon became part of IE4.0, but the technology remained niche until early 2005, when Google used it extensively in its goole groups, gmail and other interactive applications, making Ajax rapidly accepted by everyone.

The emergence of Ajax makes the transmission of data between the Client and the server much less and much faster, and also meets the needs of the initial development of the web2.0 era characterized by rich User Experience, but slowly it also exposes its drawbacks. For example, it cannot meet the requirements of real-time update data for rich interactive applications such as instant messaging. This browser-side technology is still based on the http protocol after all, and the request/response mode required by the http protocol cannot be changed unless the http protocol itself changes.

Comet: A Hack Technology (Still HTTP Protocol)

The traditional polling-based method can no longer meet the low latency requirements of web applications represented by instant messaging, and it will also bring bad User Experience. So a “server push” technology based on http long connections was hacked. This technology was named Comet, a term first proposed by Alex Russell, project director of Dojo Toolkit, in the blog post Comet: Low Latency Data for the Browser, and has been used ever since.

In fact, server push has existed for a long time and is widely used in the classic client/server model. It’s just that browsers are too lazy and do not provide good support for this technology. But the advent of Ajax made it possible to implement this technology on browsers, and the integration of Google’s gmail and gtalk first used this technology. With the resolution of some key issues (such as the loading and display issues in IE), this technology was quickly recognized, and there are now many mature open source Comet frameworks.

The following is a comparison of typical Ajax and Comet data transmission methods. The difference is simple and clear. The typical Ajax communication method is also the classic use of the http protocol. To obtain data, you must first send a request. In web applications with high Low Latency requirements, the frequency of server requests can only be increased. Comet is different. The client maintains a long connection with the server, and only when the data required by the client is updated, the server actively pushes the data to the client.

There are two main ways to implement Comet, long-polling based on Ajax and http streaming based on Iframe and htmlfile.

Long polling based on Ajax

The browser sends an XMLHttpRequest request. After receiving the request, the server side will block the request until there is data or timeout before returning it. The browser JS sends a request again after processing the request return information (timeout or valid data) to re-establish the connection. During this period, new data may have arrived on the server side, and the server will choose to save the data until the connection is re-established. The browser will retrieve all the data at once.

Streams based on Iframe and htmlfile

Iframe is an html tag. The src attribute of this tag will keep the long connection request to the specified server, and the server can keep returning data. Compared with the first method, this method is closer to the traditional server push.

In the first way, the browser will call the JS callback function directly after receiving the data, but how to respond to the data in this way? You can embed a JS script in the returned data, such as’ < script type = “text/javascript” > js_func (“data from server”) ', the server side will return the data as the parameter of the callback function, and the browser will execute this JS script after receiving the data.

WebSocket

WebScoket is a new protocol and naturally a new solution

SSE

SSE is a new protocol and naturally a new solution

Simple principle analysis of instant message scheme

Communication Principles of Traditional Web

The browser itself, as a thin Client, does not have the function of communicating directly with another Client browser in a remote place through system calls. This is different from the way our desktop applications work. Usually, desktop applications can establish a TCP connection with a process on the other end of the remote host through a socket, thus achieving full-duplex instant communication.

Since the birth of the browser, the client requests the server and the server returns the result, and there has been no change even though it has developed so far. So it is certain that in order to achieve communication between two clients, information must be forwarded through the server. For example, if A wants to communicate with B, A should first send the information to the IM application server, and the server forwards it to B according to the recipient carried in A’s information. Similarly, B to A is also in this mode.

Problems that need to be solved to realize IM application in traditional communication mode

We recognize that implementing IM software based on the web still requires the browser to request the server model. In this way, the development of IM software needs to solve the following three problems:

• Duplex communication: that is, the browser pulls (pull) server data, and the server pushes (push) data to the browser;
• Low latency: that is, the information sent by the browser A to B should be quickly forwarded to B through the server, and the information of B should also be quickly handed over to A, in fact, it requires any browser to quickly request the data of the server, and the server can Quickly push data to the browser;
• Support cross-domain: Usually the Client browser and the server are in different locations of the network, the browser itself does not allow direct access to servers under different domain names through scripts, even if the IP Address is the same and the domain name is different, and the domain name is the same and the port is different. No, this is mainly for security reasons.

Full duplex low latency solution

Client browser short polling

This is the simplest solution. The principle is that the Client sends a request to the server every short period of time through Ajax, the server returns the latest data, and then the Client updates the interface according to the obtained data, so that It indirectly realizes instant communication. The advantage is simplicity, but the disadvantage is that it puts a lot of pressure on the server and wastes bandwidth traffic (usually the data does not change).

Long polling

In the above polling solution, since a request is sent every time, the server level sends the data regardless of whether the data changes or not, and the connection is closed after the request is completed. A lot of the communication passing through this is unnecessary, so there is a long-polling method. In this way, the client sends a request to the server, the server checks if the data requested by the client has changed (whether there is the latest data), and responds immediately if there is a change, otherwise the connection is maintained and the latest data is checked regularly until a data update or connection timeout occurs. At the same time, once the client connection is disconnected, the request is issued again, which greatly reduces the number of client requests to the server in the same time.

Based on http-stream communication

There is also a communication method based on http-stream stream. The principle is to keep the client connected to the server level continuously during a request, and then the server level continuously transmits data to the client, just like the data stream, it does not send all the data to the client at once. The difference between it and the polling method is that the client only sends a request during the entire communication process, and then the server level maintains a long connection with the client, and uses this connection to send back data to the client.

This scheme is divided into several different data stream transmission methods.

Streaming method based on XHR objects

The idea of this method is to construct an XHR object, by listening to its onreadystatechange event, when its readyState is 3, get its responseText and then process it. The readyState is 3, which means that the data transmission is in progress, and the entire communication process has not yet ended. So it is still getting the data sent by the server level. It means that the data is sent until the readyState is 4, and the communication process ends. In this process, the data passed by the server level to the client is sent to the Client in the form of stream multiple times, and the Client is also obtained in the form of stream, so it is called http-streaming data stream mode.

Here, since the data received by the Client is sent in segments, it is best to define a cursor received to obtain the latest data and discard the data that has been received before, and print out the latest data received each time through this cursor., and print out the entire responseText after the communication ends.

Based on iframe data stream

Since earlier versions of IE do not allow the responseText property of XHR to be obtained when the readyState is 3, in order to achieve the use of this technology on IE, an iframe-based data stream communication method has emerged. Specifically, it is dynamically loading an iframe in the browser, so that its src attribute points to the URL of the requested server. In fact, it sends an http request to the server, and then creates a function for processing data on the browser side. At the server level, the data is output to the Client through the long connection between the iframe and the browser, but the returned data is not general data, but a function similar to ‘< script type =\ "text/javascript" > parent.process (’ “+ randomNum.toString ()+”')</ script > 'The way the script is executed, the browser receives This data will be parsed into js code and find the specified function on the page to execute, in fact, the server level indirectly uses its own data to indirectly call the client’s code, to achieve the purpose of real-time update client.

Data stream communication based on htmlfile

A new problem has arisen again. In IE, the iframe is used to request the server level. Before the server level keeps the communication connection and does not all return, the browser title has been loaded, and the bottom also shows that it is loading. This is for a product. User Experience is not good, so Google’s geniuses came up with a hack method. That is, in IE, dynamically generate an htmlfile object, which is a com component in the form of ActiveX. It is actually an HTML doc implemented in memory. By adding the generated iframe to the HTMLfile in this memory, and using the data flow of the iframe The communication method achieves the above effect. At the same time, because the HTMLfile object is not directly added to the page, it does not cause the browser to display the phenomenon of loading.

SSE

In order to solve the problem that the browser can only unidirectional transmission of data to the server level, HTML5 provides a new technology called server push event SSE, which can realize the client request server level, and then the server level uses the communication connection established with the client to push data to the client, the client receives the data and processes it. From an independent point of view, SSE technology provides the function of pushing data from the server to the browser in one way, but with the active request of the browser, it actually realizes the two-way communication between the client and the server. Its principle is to construct an eventSource object in the Client, which has the readySate property, which is represented as follows:

0: Connecting to the server.
1: The connection is opened.
2: The connection is closed.

At the same time, the eventSource object will maintain a long connection with the server, and will be automatically reconnected after disconnection. If you want to force the connection, you can call its close method. It can listen to the onmessage event, the server level follows the format of SSE data transmission to the Client, and the Client can receive data when the onmessage event is triggered, so as to perform some processing

Browser Native API

WebSocket

In the above solutions, they are all hack technologies formed by combining the browser’s one-way request to the server or the server’s one-way push of data to the browser. In HTML5, in order to enhance the functionality of the web, websocket technology is provided. It is not only a web communication method, but also an application layer protocol. It provides native dual full-duplex cross-domain communication between the browser and the server. Through the establishment of a websocket connection (actually a TCP connection) between the browser and the server, it can achieve Client-to-server and server-to-client data transmission at the same time. For the principle of this technology, please refer to: “WebSocket Detailed Explanation (1): Preliminary Understanding of WebSocket Technology”, “WebSocket Detailed Explanation (2): Technical Principle, Code Demonstration and Application Cases”, “WebSocket Detailed Explanation (3): In-depth WebSocket Communication Protocol Details”, I will not go into details here, and give the code directly. Before looking at the code, you need to understand the entire working process of websocket.

The first is Client new, a websocket object, which will send an http request to the server level. The server level finds that this is a webscoket request, and will agree to the protocol conversion and send back a response with a 101 status code to the Client. The above process is called a handshake. After this handshake, the Client establishes a TCP connection with the server level. On this connection, the server level and the Client can communicate in two directions. At this time, the two-way communication at the application layer is the ws or wss protocol, which has nothing to do with http. The so-called WS protocol requires the Client and server levels to send data messages network packets (frames) in a certain format before the other party can understand.

Reference article:
Web端即时通讯技术盘点：短轮询、Comet、Websocket、SSE
新手入门贴：史上最全Web端即时通讯技术原理详解

Basic knowledge of linear algebra

Vector

1. A vector is a line segment with a direction and a length, written in mathematical notation: $\ vec {a} $
2. Or expressed as a line segment from the beginning to the end, for example, there are two points A, B, then the vector from A to B can be expressed as $\ vec {AB} = B - A $
3. Vectors have no absolute starting point, that is, if a vector is moved in space, the vector itself remains unchanged
4. The length of the vector is expressed as: $| |\ vec {a} | | $, and the vector divided by its own length is the unit vector of changing direction: $\ frac {\ vec {a}} {| |\ vec {a} | |} $
5. The addition of vectors conforms to the parallelogram rule or the triangle rule

Coordinates can be represented by vectors

If there are two unit vectors perpendicular to each other, and we choose one of them to be $\ vec {X} $and the other to be $\ vec {Y} $, we can represent a vector $\ vec {A} =\ binom {x} {y} $, or $\ vec {A} ^ T = (x, y) $

We assume that the starting point of this vector is the origin, then the end point is the (x, y) point we usually say in geometry

Multiplication of a vector

Vectors can be multiplied by dot. Suppose there are two vectors $\ vec {a},\ vec {b} $in different directions. Suppose their starting points are put together, and there will be an angle between them, assuming $\ theta $

那么 $\vec{a} \cdot \vec{b} = ||\vec{a}||||\vec{b}||cos\theta$

The point product of a vector has commutative and associative laws.

$$\vec{a} \cdot \vec{b} = \vec{b} \cdot \vec{a} \\\vec{a} \cdot (\vec{b} + \vec{c}) = \vec{a} \cdot \vec{b} + \vec{a} \cdot \vec{c} \\(k\vec{a}) \cdot \vec{b} = \vec{a} \cdot (k\vec{b}) = k(\vec{a} \cdot \vec{b})$$

The vector points are multiplied by coordinates:

$$\ vec {a}\ cdot\ vec {b} =\ binom {x _ a} {y _ a}\ cdot\ binom {x _ b} {y _ b} = x _ ax _ b + y _ ay _ b\\\vec{a} \cdot \vec{b} = \begin{bmatrix} x_a \\ y_a \\ z_a\end{bmatrix} \cdot \begin{bmatrix} x_b \\ y_b \\ z_b\end{bmatrix} = x_ay_a + x_by_b + z_az_b$$

Applications of vector dot multiplication are:

• We can use the result of the dot product between the two vectors and the 0 comparison to determine whether the angle between the two is acute or obtuse.
• Find the angle between two vectors
• Find the projection of one vector on another vector

Cross product of vectors

Another multiplication of vectors is cross multiplication

The direction of the cross product of the vector follows the right-hand rule. Assuming $\ vec {a}\ times\ vec {b} $, then the direction of the result is that the four fingers of the right hand turn from the direction of the vector a to the direction of b, clench tightly, and then give a thumbs up is, the direction of the thumb. That is, the result of the cross product is perpendicular to the plane where a and b are located

Then the length of the cross product is actually $|\ vec {a}\ times\ vec {b }||=||\ vec {a} | | | |\ vec {b} | | sin\ theta $

The cross product of vectors does not support associativity. To be precise, the result of the order of commutative cross products is reversed, that is, the direction is reversed: $\ vec {a}\ times\ vec {b} = -\ vec {b}\ times\ vec {a} $

The cross product of the vector is represented by a matrix:

$$\vec{a} \times \vec{b} = \begin{bmatrix} y_az_b - y_bz_a \\ x_az_b - x_bz_a \\ x_ay_b - x_by_a\end{bmatrix} \\\vec{a} \times \vec{b} = A * \vec{b} = \begin{bmatrix} 0 & -z_a & y_a \\ z_a & 0 & -x_a \\ -y_a & x_a & 0 \\\end{bmatrix} \begin{bmatrix} x_b \\ y_b \\ z_b\end{bmatrix}$$

Cross product can determine whether a vector is on the left or right of another vector. This is easier to understand. When the right-hand rule rotates, it rotates clockwise or counterclockwise, and the result is the opposite.

Another function is to determine whether a point is inside a triangle

In the figure above, if the symbols of vector BC cross-multiplied by vector BP, vector CA cross-multiplied by vector CP, and vector AB cross-multiplied by vector AP are all the same, then it means that the P point is inside

Matrix

A matrix is a two-dimensional array of m rows and n columns.

The premise that two matrices can be multiplied is that the number of columns in the first matrix and the number of rows in the second matrix are the same.

That is, a matrix with M rows and N columns can be multiplied by a matrix with N rows and P columns, resulting in a matrix with M rows and P columns.

Suppose A and B are multiplied by two matrices to a matrix C, where each term of A is $a_ {ij} $, each term of B is $B_ {ij} $, and each term of C is $c_ {ij} $, then $c_ {ij} =\ sum_ {k = 0} ^ {k = N} a_ {ik} b_ {kj} $

The important point here is how to write the equations of coordinate transformation in the form of matrices

For example, how to say that points in a two-dimensional coordinate system are symmetrical according to the y-axis

Just write it in a system of equations

$$\begin{cases} x' = -x; \\ y' = y\end{cases}$$

Write it in matrix form

$$\begin{bmatrix} -1 & 0 \\ 0 & 1\end{bmatrix}\begin{bmatrix} x \\ y\end{bmatrix} = \begin{bmatrix} -x \\ y\end{bmatrix}$$

Each matrix has its own transpose matrix and inverse matrix.

The transpose matrix of A is written as $A_T $, and $ (AB) ^ T = B ^ TA ^ T $

The Inverse Matrix of A is written as $A ^ {-1} $, and $AA ^ {-1} = I $, where I is the identity matrix, and any matrix multiplied by the identity matrix equals nothing, that is, any matrix multiplied by A The change produced by the matrix can be restored by multiplying the Inverse Matrix of A

How to use a matrix to transform (Transform)

2D transformation

1. Zoom out

When the image is scaled by s times, it is represented by the equation

$$\begin{cases}x' = sx \\y' = sy \\\end{cases}$$

The corresponding scaling matrix is:

$$\begin{bmatrix}s & 0 \\0 & s\end{bmatrix}$$

2. Inversion

$$\begin{bmatrix}-1 & 0 \\0 & 1 \\\end{bmatrix}$$

3. Shear

$$\begin{bmatrix}1 & a \\0 & 1\end{bmatrix}$$

Step 4 Rotate

$$\begin{bmatrix}cos\theta & -sin\theta \\sin\theta & cos\theta\end{bmatrix}$$

So far, all our transformations can be expressed in matrix form, because our previous transformations can be expressed in the following equation:

$$\begin{cases}x' = ax + by \\y '= cx + dy\end{cases}$$

Expressed as a matrix is

$$\begin{bmatrix}x' \\ y'\end{bmatrix}=\begin{bmatrix}a & b \\c & d \\\end{bmatrix}\begin{bmatrix}x \\ y\end{bmatrix}$$

But the problem is that you can’t represent translation in this way, because translation can’t be written in this form

5. Translation

The system of equations for translation is like this:

$$\begin{cases}x' = x + t_x;\\y' = y + t_y;\end{cases}$$

If scaling, rotation, and translation are all represented by matrices, they should be as follows:

$$\begin{bmatrix}x' \\ y'\end{bmatrix}=\begin{bmatrix}a & b \\c & d \end{bmatrix}\begin{bmatrix}x \\ y\end{bmatrix}+\begin{bmatrix}t_x \\ t_y\end{bmatrix}$$

So at this time we have to introduce homogeneous coordinates, that is, add w. At this time, the 2D point coordinates are represented by (x, y, 1), and the 2D vector is represented by (x, y, 0)

When representing a point, w is 1, when representing a vector, w is 0, and there is a very magical place like this, that is, if two points are subtracted, w will become 0, which happens to be a vector, point and vector Adding, w is 1, which is also a point

Using homogeneous coordinates, we can uniformly rotate, scale, and translate into a matrix

The translation is expressed in homogeneous coordinates as:

$$\begin{bmatrix}x' \\ y' \\ z'\end{bmatrix}=\begin{bmatrix}1 & 0 & t_x \\0 & 1 & t_y \\0 & 0 & 1 \\\end{bmatrix}\begin{bmatrix}x \\ y \\ 1\end{bmatrix}=\begin{bmatrix}x + t_x \\y + t_y \\1\end{bmatrix}$$

6. Mixing

We can now use homogeneous coordinates to represent rotation, translation, and scaling respectively

$$S(s_x, s_y) = \begin{bmatrix}s_x & 0 & 0 \\0 & s_y & 0 \\0 & 0 & 1\end{bmatrix}\\R(\theta) = \begin{matrix}cos\theta & -sin\theta & 0 \\sin\theta & cos\theta & 0 \\0 & 0 & 1\end{matrix}\\T(t_x, t_y) = \begin{bmatrix}1 & 0 & t_x \\0 & 1 & t_y \\0 & 0 & 1\end{bmatrix}$$

So how do we mix these operations?

As you can see in the figure above, we now translate and then rotate. The result of translation is different from that of rotation first, because our rotation matrix is rotated around the origin

Therefore, we generally specify the order to perform mixing operations. The first operation is multiplied left by the coordinates of the current point, and after obtaining the new point, it is multiplied left by the next operation matrix.

Then, although our order cannot be changed because matrix multiplication has no commutativity, matrix multiplication has associativity, that is:

So we can achieve the result of multiplying the rotation, scaling, and translation matrices as the transformation matrix, and then multiplying it left with each point

There is actually another problem here, which is, what if we just want a point to rotate around its lower left corner?

It’s simple, translate the bottom left corner to the origin, then rotate, and finally translate the bottom left corner back:

3D transformation

The transformation of 3D is actually no different from 2D, except that homogeneous coordinates have four dimensions

The transformation matrix is as follows:

$$\begin{bmatrix}x' \\ y' \\ z' \\ 1\end{bmatrix}=\begin{bmatrix}a & b & c & t_x \\d & e & f & t_y \\g & h & o & t_z \\0 & 0 & 0 & 1\end{bmatrix}\begin{bmatrix}x \\ y \\ z \\ 1\end{bmatrix}$$

The scaling matrix can be written as;

$$S(s_x, s_y, s_z) = \begin{bmatrix}s_x & 0 & 0 & 0\\0 & x_y & 0 & 0\\0 & 0 & s_z & 0 \\0 & 0 & 0 & 1\end{bmatrix}$$

The translation matrix can be expressed as:

$$T(t_x, t_y, t_z) = \begin{bmatrix}1 & 0 & 0 & t_x \\0 & 1 & 0 & t_y \\0 & 0 & 1 & t_z \\0 & 0 & 0 & 1\end{bmatrix}$$

Rotation is more complicated because it can be divided into rotation around different axes

$$R_x(\theta) = \begin{bmatrix}1 & 0 & 0 & 0 \\0 & cos\theta & -sin\theta & 0 \\0 & sin\theta & cos\theta & 0 \\0 & 0 & 0 & 1\end{bmatrix} \\R_y(\theta) = \begin{bmatrix}cos\theta & 0 & sin\theta & 0 \\0 & 1 & 0 & 0 \\sin\theta & 0 & cos\theta & 0 \\0 & 0 & 0 & 1\end{bmatrix} \\R_z(\theta) = \begin{bmatrix}cos\theta & -sin\theta & 0 & 0 \\sin\theta & cos\theta & 0 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{bmatrix}$$

There is also a formula to multiply any rotation matrix Factorization by the rotation matrix in three axis directions. The following way represents the rotation angle of $\ theta $around the $\ vec {n} $axis.

$$R(\vec{n}, \alpha) = cos\alpha\vec{I} + (1 - cos\alpha)\vec{n}\vec{n}^T + sin\alpha\begin{bmatrix}0 & -n_z & n_y \\n_z & 0 & -n_x \\-n_y & n_x & 0 \\\end{bmatrix}$$

Observation Transformation in Graphics

When we are doing game development and writing Shader, we often use something called an MVP matrix to change the coordinates of points on the model to the coordinates on the screen.

The MVP here refers to Model, View, and Projection, which means model transformation, view transformation, and projection transformation

Model transformation is to change the coordinates from the coordinate system of the model itself to the coordinates of the game world coordinate system

View transformation is to change the coordinates on the world coordinate system into the coordinates of the observation space

The projection transformation is to change the coordinates of the observation space into the clipping space. In fact, this step does not do the operation of projecting to the two-dimensional plane. The specific projection operation is written in the GPU in the rendering pipeline, and is generally not processed in the Shader.

The observation transformation we are talking about here is just a step in the rendering pipeline. At the beginning, it is operated vertex-by-vertex in the vertex shader. We get the coordinates of the point in the clipping space. In the next rendering pipeline, we have to go through grating., sampling, chip-by-chip shader output color, depth test, etc. will finally be projected onto a two-dimensional plane.

Tired than the usual way we take pictures, the Model matrix is like we find a suitable camera position, the View matrix is to use the camera to find an angle

We don’t talk about the Model matrix here, because it is the same as the View matrix, which changes from one coordinate system to another, and the Projection matrix is different in that the observation space is a box with a side length of 1 cube, we need to consider scaling

View matrix

View matrix is the point from the world coordinates into the observation space, i.e., from the origin of the world coordinate system relative to the camera position becomes coordinates.

After the Model transformation, we have the coordinates of the point in space, now we need to define the position and orientation of the camera in space:

The coordinates and orientation of the camera here are relative to the world coordinate system. And our object coordinates are currently relative to the world coordinate system.

Now all we have to do is change the object coordinates relative to the world coordinate system to relative to the camera coordinates.

Here we introduce a common physical concept - relative motion, that is, if the same transformation operation is performed on the camera and the object, the relative position of the two remains unchanged.

Then we can now try to move the camera to the origin. The observation direction of the camera is towards the negative direction of the z-axis of the world coordinate system, find the matrix of this transformation, and then apply this matrix to each point, which is equivalent to moving the object to the observation space. Although from the point of view of the object coordinates, a certain movement is made in the world coordinate system, this movement does not change the relative position of the object and the camera, and also successfully moves the camera to the origin of the world coordinates, so the result is equal to moving the object to the observation space.

So how do you get this view matrix?

This method is more complicated

We can use a better property here, that is, the rotation matrix is actually an orthogonal matrix, the Inverse Matrix and the transpose matrix of the orthogonal matrix are the same, that is, we can find the matrix of the world coordinate axis transformed into the camera coordinate axis, and then find his transpose matrix, that is, the Inverse Matrix, which is the matrix of the camera coordinate axis transformed into the world coordinate axis

Projection matrix

Just through the View matrix, the relative position of our camera and the object as a whole remains unchanged and moves to the position of the camera at the origin of the world coordinate system.

What is the purpose of this? Of course, there is an advantage that it is easy to understand, but in fact, it does not make sense for a calculator, because it is all multiplied by a matrix, and the amount of calculation will not make a difference.

Another advantage of this is to reduce the calculation of the projection matrix.

Our projection matrix is divided into two types, one is parallel projection and the other is orthogonal projection:

Parallel projection

Let’s first look at the relatively simple parallel projection

A relatively simple way to understand this projection is to just throw away the z-axis, which is the coordinate of the final point on the screen, and then both the x and y directions are translated and scaled between [-1, 1]. The reason why we can just throw away z here is that our camera is moved to the origin and in the negative direction of the z-axis.

However, dropping the z-axis cannot be done yet. We still need the information of z to do in-depth tests later. What we need to do now is to normalize x, y, z to the cube of $[-1,1] ^ 3 $

As shown in the figure below, in the parallel projection, we start with a cube in the observation space. We need to move the center of this cube to the origin and scale it to a cube with side length 1

Orthogonal projection

The difference between orthogonal projection and parallel projection is that there is a near-large and far-small effect, and its initial observation space is not a cube, but a ladder

We are looking for this ladder normalization matrix to be divided into two steps:

1. Scale the ladder to a cube, and the observation space at this time is equivalent to a parallel projection
2. Reuse normalized matrices with parallel projections

So we’re focusing on the first step right now

And because our current camera is facing the negative direction of z, so the zoom bottom will not affect the z coordinate, we only need to focus on x and y, we take the y coordinate for example:

Similarly, the left side of x becomes $x '=\ frac {n} {z} x $

Then at this time, our transformation matrix can be written. First, the transformation of the orthogonal projection space into the parallel projection space

$$M_{presp-ortho}\begin{bmatrix}x \\ y \\ z \\ 1\end{bmatrix} = \begin{bmatrix}\ frac {n} {z} x\\ frac {n} {z} y\ z\\ 1\end{bmatrix} = \begin{bmatrix}nx \\ ny \\ z^2 \\ z\end{bmatrix}\\That is\\M_{presp-ortho} = \begin{bmatrix}n & 0 & 0 & 0 \\0 & n & 0 & 0 \\0 & 0 & z & 0 \\0 & 0 & 1 & 0 \\\end{bmatrix}$$

Then the formal orthogonal projection View matrix is $M_ {press-ortho} $multiplied by the parallel projection View matrix

At this point, we have completed the transformation of a point from model space to normalized space in the world coordinate system

This code uses an old library called canvas-to-svg

This library simulates canvas and exposes some canvas apis, which can finally be exported as svg

It should be noted that some canvas APIs do not support, such as setTransform, resetTransform, etc. If you need to use these two APIs, you can split them into translate and rotate, etc

Directly upload the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

import CanvasToSVG from 'canvas-to-svg';
function addTextWaterMaskToCanvas(
  context: CanvasRenderingContext2D,
  textWaterMask: string,
) {
  context.font = '16px Microsoft YaHei';
  context.fillStyle = '#F2F3F5';
  const { width, height } = context.canvas;
  const textMaskCountHorizontal = Math.max(1, width / 150);
  const textMaskCountVertical = Math.max(1, height / 150);
  const fwidth = width / textMaskCountHorizontal;
  const fheight = height / textMaskCountVertical;
  for (let hh = 0; hh < height; hh += fheight) {
    for (let ww = 0; ww < width; ww += fwidth) {
      context.translate(ww, hh);
      context.rotate(-Math.atan(height / width));
      context.fillText(textWaterMask, -fwidth / 2, fheight / 2);
      context.rotate(Math.atan(height / width));
      context.translate(-ww, -hh);
    }
  }
}

export function downloadFullImageWithWaterMaskCreator(
  graphInstance: IGraph | null,
  textWaterMask: string,
) {
  return function (
    name?: string,
    type?: DataUrlType,
    imageConfig?: {
      backgroundColor?: string;
      padding?: number | number[];
    },
    callback?: (success: boolean) => void,
  ) {
    graphInstance?.toFullDataURL(
      res => {
        const image = new Image();
        image.src = res;
        image.onload = function () {
          try {
            // const canvas = document.createElement('canvas');
            // canvas.width = image.width;
            // canvas.height = image.height;
            // const context = canvas.getContext('2d');
            const context = new CanvasToSVG(image.width, image.height);
            if (context && name) {
              context.rect(0, 0, image.width, image.height);
              context.drawImage(image, 0, 0, image.width, image.height);

              addTextWaterMaskToCanvas(context, textWaterMask);

              // downloadCanvasImage(canvas, name, type);

              const svg = context.getSerializedSvg(true);
              const svgData = URL.createObjectURL(
                new Blob([svg], { type: 'image/svg' }),
              );
              const downloadLink = document.createElement('a');
              downloadLink.download = `${name}.svg`;
              downloadLink.href = svgData;
              downloadLink.click();

              callback?.(true);
            }
          } catch {
            callback?.(false);
          }
        };

        image.onerror = function () {
          callback?.(false);
        };
      },
      type,
      imageConfig,
    );
  };
}

The reason why these two algorithms are considered together is that the simulated annealing algorithm is like an enhanced version of the greedy algorithm.

If the local optimal solution of each step of a problem cannot obtain the global optimal solution, the algorithm thinking of our conventional front-end and back-end programmers is Dynamic Programming.

But for AI development, the parameters may be hundreds of millions of levels, there is no way Dynamic Programming, this time you can try to use simulated annealing algorithm to jump out of this part with a certain probability, to find a better answer in the whole, but in fact This algorithm is not like a conventional algorithm, and will get an inevitable result.

This article summarizes and compares the two algorithms together to see where they are applicable and how to use them.

Greedy algorithm

What is a greedy algorithm

The essence of greed is to choose the local optimum of each stage, so as to achieve the global optimum.

This is a bit abstract, let’s take an example:

For example, if you have a pile of banknotes, you can take ten of them. If you want to reach the maximum amount, how do you take them?

Specify that you take the largest amount each time, and the end result is to take the largest amount of money.

Each time you take the largest is the local optimal, and finally take the largest amount of money is to launch the global optimal.

As another example, if there are a bunch of boxes, and you have a backpack with a volume of n, how to fill the backpack as much as possible, if you still choose the largest box every time, it will not work. At this time, Dynamic Programming is needed.

When to use a greedy algorithm

To be honest, there is no fixed routine for greedy algorithms.

So the only difficulty is how to use the local optimum to derive the overall optimum.

So how can we see if the local optimum can lead to the overall optimum? Are there any fixed strategies or routines?

Sorry, no! Manually simulate by yourself. If the simulation is feasible, you can try the greedy strategy. If it is not feasible, Dynamic Programming may be required.

A colleague asked how to verify whether a greedy algorithm can be used?

The best strategy is to give counterexamples. If you can’t think of counterexamples, try greed.

Greedy algorithm general steps:

• Break down the problem into several sub-problems
• Find the right greedy strategy
• Solve the optimal solution for each sub-problem
• Stack locally optimal solutions into globally optimal solutions

These four steps are actually too theoretical. We usually do greedy topics, and it is difficult to think according to these four steps. It is really a bit “chicken ribs”.

When doing the question, as long as you think clearly, what is the local optimal, if you deduce the global optimal, it is actually enough.

Simulated annealing algorithm

Metal annealing principle

Metal annealing is a metal heat treatment process in which the metal is heated to a certain temperature, kept for a sufficient time, and then cooled at a suitable speed (usually slow cooling, sometimes controlled cooling). The simulated annealing algorithm is derived from the principle of solid annealing, which heats the solid to a sufficiently high temperature, and then allows it to cool slowly. When heated, the internal particles of the solid become disordered with the temperature rise, and the internal energy increases, while when slowly cooled, the particles gradually become Orderly, reaching an equilibrium state at each temperature, and finally reaching the ground state at room temperature, the internal energy is minimized.

When in a low temperature state, the molecules in the solid have very low internal energy and vibrate in a small range in their original position. If the solid is heated to a certain temperature, the internal energy of the molecules will increase, the thermal movement will intensify, and the disorder of the molecular arrangement will increase. At this time, the temperature is slowly lowered, and at each temperature an equilibrium state (that is, a quasi-static process) is reached, the energy of the molecules gradually decreases, and eventually returns to the state of orderly arrangement, and the internal energy of the molecules also drops to a minimum.

Simulated annealing algorithm

The earliest idea of simulated annealing algorithm (Simulated Annealing, SA) was proposed by N. Metropolis et al in 1953. In 1983, S. Kirkpatrick et al successfully introduced the annealing idea into the field of combinatorial optimization. It is a stochastic optimization algorithm based on Monte-Carlo iterative solution strategy, and its starting point is based on the similarity between the annealing process of solid matter in physics and general combinatorial optimization problems.

Before introducing simulated annealing, it is necessary to introduce the mountain climbing algorithm.

Hill climbing algorithm

The hill climbing algorithm is a simple greedy search algorithm, which selects an optimal solution from the adjacent solution space of the current solution as the current solution every time until a local optimal solution is reached.

The implementation of the hill-climbing algorithm is very simple, and its main disadvantage is that it will fall into the local optimal solution, and it may not be able to search for the global optimal solution. As shown in the figure above: Assuming that point C is the current solution, the hill-climbing algorithm will stop searching when it searches for the local optimal solution at point A, because no matter where point A moves in that direction, it cannot get a better solution.

Core idea of simulated annealing

Simulated annealing is actually a greedy algorithm, but its search process introduces random factors. The simulated annealing algorithm accepts a solution that is worse than the current solution with a certain probability, so it is possible to jump out of this local optimal solution and achieve a global optimal solution.

The simulated annealing algorithm starts from a higher initial temperature, with the continuous decline of temperature parameters, combined with certain probability sudden jump characteristics, randomly finds the global optimal solution of the target function in the solution space, that is, the local optimal solution can probabilistically jump out and eventually tend to the global optimal.

The calculation of “certain probability” here refers to the annealing process of metal smelting, which is also the origin of the name of the simulated annealing algorithm. The temperature T is regarded as the control parameter, the target function value f is regarded as the internal energy E, and a state of a solid at a certain temperature T corresponds to a solution
Then the algorithm attempts to reduce the target function f (internal energy E) as the control parameter T decreases, until it tends to the global minimum (the lowest energy state at low temperature in annealing), just like the metal annealing process.

Mathematical principles of simulated annealing

From the above, we know that the global optimal solution of the target function will be randomly found in the solution space in combination with the probability sudden jump characteristic. So what is the specific mechanism for updating the solution? If the new solution is better than the current solution, the new solution is accepted, otherwise it is judged whether to accept the new solution based on the Metropolis criterion. The acceptance probability is:

\begin{equation}P = \begin{cases} 1 & E_{t + 1} < E_t;\\ e^{-\frac{E_{t + 1} - E_t}{kT}} & E_{t + 1} \geq E_t;\end{cases}\end{equation}

Assuming that the solution of the search at the current time is $x_t $, and the corresponding system energy (target function) is $E_t $, a random perturbation is applied to the search point to generate a new solution $x_ {t + 1} $, and accordingly, the system energy is $E_ {t + 1} $, then the acceptance probability of the system from the search point to the transition is the above formula.

That is, if the energy corresponding to the new solution is lower, then the probability of acceptance is 1, that is, it must be accepted. If the energy corresponding to the new solution is high, then accept the new solution with the probability of $e ^ {-\ frac {E_ {t + 1} - E_t} {kT}} $, that is, use this probability to jump out of this local optimal solution.

This process is to randomly select x, and then find a corresponding E lowest x out, where E is lower corresponding to our specific algorithm is closer to the goal we need

There is also a problem here, which is how to choose the size of each change in x when we randomly select x.

This problem is actually the origin of the word’annealing ', that is, the temperature will gradually decrease, and the magnitude of x change will become smaller and smaller.

Simulated annealing process

The essence of the algorithm is divided into two layers of cycles. At any temperature level, random disturbances generate a new solution, and calculate the change of the target function value to decide whether to accept it. Since the initial temperature of the algorithm is relatively high, the new solution that increases E may also be accepted initially, so it can jump out of the local minimum, and then by slowly reducing the temperature, the algorithm may eventually converge to the global optimal solution. The specific process is:

1. Let $T = T_0 $, representing the initial temperature at which the annealing starts, randomly generate an initial solution $x_0 $, and calculate the corresponding target function value $E_0 $;
2. Let $T = kT $, where k is between 0 and 1, which is the rate of temperature drop;
3. Apply random perturbation to the current solution $x_t $, generate a new solution $x_ {t + 1} $in its neighborhood, and calculate the corresponding target function value $E_ {t + 1} $$\Delta E =E_{t+1} - E_t$
4. If $\ Delta E < 0 $accepts the new solution as the current solution, otherwise it is judged whether to accept the new solution according to the probability $e ^ {-\ frac {\ Delta E} {kT}} $;
5. Repeat the disturbance and acceptance process L times at temperature T, i.e. perform steps 3 and 4;
6. Determine whether the temperature reaches the termination temperature level, if so, terminate the algorithm, otherwise return to step 2.

There are a few points to note:

• The selection of the initial point has a certain impact on the results of the algorithm, it is best to run multiple times to make a comprehensive judgment on the results.
• In the early stage of algorithm operation, the temperature drops quickly to avoid accepting too many poor results. As the running time increases, the temperature drop slows down to stabilize the results faster.
• When the number of iterations increases to a certain number, the result may have reached stability, but there is still some time before the algorithm ends. Appropriate output conditions should be added when designing the program, and the program can be ended if the output conditions are met.

Application of simulated annealing

• The application of simulated annealing algorithm in VLSI design, using simulated annealing algorithm to optimize the design of VLSI (Very Large Scale Integration, Very Large Scale Integrated Circuit), is one of the most successful application examples of simulated annealing algorithm at present. The simulated annealing algorithm can almost complete all optimized VLSI design work well. Such as global wiring, board layout, layout and logic minimization, etc.

• Simulated annealing algorithm can be used for image restoration and other work, that is, to restore a contaminated image into a clear original image and filter out the distorted part. Therefore, its application prospect in image processing is broad.

• Application of simulated annealing algorithm in neural network computer. The simulated annealing algorithm has the ability to jump out of the trap of local optima. In the Boltzmann machine, even if the system falls into the trap of local optima, after a period of time, it can jump out again, and the system will eventually converge in the direction of the global optimal value.

• In the force-oriented layout algorithm, the idea of simulated annealing is actually implicit. At the beginning, all nodes are randomly arranged, and then the position of the node changes due to the action of the force between the nodes. The position change is equivalent to the x change, and the changed position, if the resultant force received by the node decreases, which is equivalent to a decrease in energy E, we accept the new solution, and then as the resultant force decreases, it is actually equivalent to a decrease in T, so the impact on the position will become smaller, the full resultant force decreases below a certain threshold, and the position change is not large, and we think the algorithm converges to the result.

This thing looks like a new DSL or something, providing many commonly used and built-in CSS styles, encapsulating them into syntax, and then providing a PostCSS plugin to process the encapsulated syntax back into ordinary CSS.

All we need to do is install Tailwind’s PostCSS plugin and use his encapsulated DSL.

What is PostCSS?

In web application development, the writing of CSS code is an important part. From the original CSS1 to the current CSS3, and then to the next version of the CSS specification, the specification itself has been constantly evolving. This has brought efficiency improvements to developers. However, similar to the situation of specifications in other web fields, CSS specifications have always had various problems in terms of browser compatibility. The progress of implementing CSS specifications in different browsers also varies greatly. In addition, there is still a certain gap between the development speed of the CSS specification itself and the expectations of the community. This is also an important reason why CSS preprocessing languages such as SASS and LESS can become popular. SASS and LESS provide many more practical features and also reflect the needs of developers for CSS languages. PostCSS, which is introduced in this article, is a popular tool for processing CSS. PostCSS adds endless possibilities to CSS processing with its powerful plugin system.

PostCSS itself is a relatively single-function tool. It provides a way to handle CSS with JavaScript code. It is responsible for parsing CSS code into an Abstract Syntax Tree (AST) structure, which is then handled by plugins. Plugins can perform various operations based on the AST of CSS code, such as supporting variables and mixins, increasing browser-related declaration prefixes, or transpiling style rules using future CSS specifications into formats supported by current CSS specifications. From this perspective, the strength of PostCSS lies in its constantly evolving plugin system. Currently, PostCSS has more than 200 plugins with different functions. Developers can also develop their own PostCSS plugins according to the needs of the project.

** PostCSS has brought controversy over its classification in the community since its birth. This is mainly due to the post in its name, which is easily reminiscent of PostCSS being used as a post-processor for CSS, thus being compared to existing CSS pre-processor languages such as SASS and LESS. Analogy **.

In fact, there are only two main functions of PostCSS: the first is to parse CSS into an AST that JavaScript can manipulate, and the second is to call a plugin to process the AST and get the result. Therefore, PostCSS cannot simply be classified as a CSS preprocessing or post-processing tool. PostCSS can perform many tasks, covering both preprocessing and post-processing in the traditional sense. PostCSS is a brand new tool that brings front-end developers a different way of handling CSS.

How to use PostCSS

PostCSS is generally not used alone, but integrated with existing build tools. PostCSS can be integrated with mainstream build tools such as Webpack, Grunt, and Gulp. After completing the integration, select the PostCSS plugin that meets the functional requirements and configure it.

Postcss-loader is used in Webpack to perform plug-in processing. In Listing 1, postcss-loader is used to process .css files, added after style-loader and css-loader. An additional postcss method returns the required PostCSS plug-in. The role of require (‘autoprefixer’) is to load the Autoprefixer plug-in.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

var path = require('path');

module.exports = {
context: path.join(__dirname, 'app'),
entry: './app',
output: {
path: path.join(__dirname, 'dist'),
filename: 'bundle.js'
},
module: {
loaders: [
{
test: /\.css$/,
loader: "style-loader!css-loader!postcss-loader"
}
]
},
postcss: function () {
return [require('autoprefixer')];
}
}

How to enable and set up Tailwind

passed

For most projects (and to take advantage of Tailwind’s custom features), you will need to install Tailwind and its dependencies via npm.

1

npm install -D tailwindcss@latest postcss@latest autoprefixer@latest

Since Tailwind does not automatically add browser engine prefixes to the generated CSS, we recommend installing autoprefixer to handle this issue, as shown in the code snippet above.

As

Add tailwindcss and autoprefixer to your PostCSS configuration. In most cases, this is the postcss.config.js file in the root of the project, but it may also be the .postcssrc file or specified by the postcss key in the package.json file.

1
2
3
4
5
6
7

// postcss.config.js
module.exports = {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  }
}

Create your profile

If you want to customize your Tailwind installation, you can generate a configuration file using the Tailwind CLI tool, which is included in the npm package tailwindcss.

1

npx tailwindcss init

This will create a minimal tailwind.config.js file in your project root directory.

1
2
3
4
5
6
7
8
9
10